{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-02-12T22:17:19.603440Z",
     "start_time": "2019-02-12T22:17:19.523397Z"
    }
   },
   "source": [
    "<h1><center> Facial Emotion Recognition - Pre-processing with EmotionalDAN</center></h1>\n",
    "<center> A project for the French Employment Agency </center>\n",
    "<center> Telecom ParisTech 2018-2019 </center>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# I. Context"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The aim of this notebook is to explore facial emotion recognition techniques from a live webcam video stream. \n",
    "\n",
    "The data set used for training is the Kaggle FER2013 emotion recognition data set : https://www.kaggle.com/c/challenges-in-representation-learning-facial-expression-recognition-challenge/data\n",
    "\n",
    "The models explored include :\n",
    "- Manual filters \n",
    "- Deep Learning Architectures\n",
    "- DenseNet Inspired Architectures\n",
    "\n",
    "This model will be combined with voice emotion recongition as well as psychological traits extracted from text inputs, and should provide a benchmark and a deep analysis of both verbal and non-verbal insights for candidates seeking for a job and their performance during an interview."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# II. General imports"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Versions used :"
   ]
  },
  {
   "cell_type": "raw",
   "metadata": {},
   "source": [
    "Python : 3.6.5\n",
    "Tensorflow : 1.10.1\n",
    "Keras : 2.2.2\n",
    "Numpy : 1.15.4\n",
    "OpenCV : 4.0.0"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-03-06T07:44:28.399172Z",
     "start_time": "2019-03-06T07:44:19.771919Z"
    }
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/anaconda3/lib/python3.6/site-packages/h5py/__init__.py:36: FutureWarning: Conversion of the second argument of issubdtype from `float` to `np.floating` is deprecated. In future, it will be treated as `np.float64 == np.dtype(float).type`.\n",
      "  from ._conv import register_converters as _register_converters\n",
      "Using TensorFlow backend.\n",
      "/anaconda3/lib/python3.6/site-packages/requests/__init__.py:80: RequestsDependencyWarning: urllib3 (1.21.1) or chardet (2.3.0) doesn't match a supported version!\n",
      "  RequestsDependencyWarning)\n"
     ]
    }
   ],
   "source": [
    "### General imports ###\n",
    "import numpy as np\n",
    "import pandas as pd\n",
    "import matplotlib.pyplot as plt\n",
    "from time import time\n",
    "from time import sleep\n",
    "import re\n",
    "import os\n",
    "import argparse\n",
    "from collections import OrderedDict\n",
    "import matplotlib.animation as animation\n",
    "\n",
    "### Image processing ###\n",
    "from scipy.ndimage import zoom\n",
    "from scipy.spatial import distance\n",
    "import imutils\n",
    "from scipy import ndimage\n",
    "import cv2\n",
    "import dlib\n",
    "from __future__ import division\n",
    "from imutils import face_utils\n",
    "\n",
    "### CNN models ###\n",
    "import keras\n",
    "from keras.preprocessing.image import ImageDataGenerator, array_to_img, img_to_array, load_img\n",
    "from keras.callbacks import TensorBoard\n",
    "from keras.models import Sequential\n",
    "from keras.layers.core import Dense, Dropout, Activation, Flatten\n",
    "from keras.layers.convolutional import Conv2D, MaxPooling2D, SeparableConv2D\n",
    "from keras.utils import np_utils\n",
    "from keras.regularizers import l2#, activity_l2\n",
    "from keras.optimizers import SGD, RMSprop\n",
    "from keras.utils import to_categorical\n",
    "from keras.layers.normalization import BatchNormalization\n",
    "from keras import models\n",
    "from keras.utils.vis_utils import plot_model\n",
    "from keras.layers import Input, GlobalAveragePooling2D\n",
    "from keras.models import Model\n",
    "from tensorflow.keras import layers\n",
    "\n",
    "### Build SVM models ###\n",
    "from sklearn.preprocessing import OneHotEncoder\n",
    "from sklearn.model_selection import train_test_split\n",
    "from sklearn.metrics import accuracy_score\n",
    "from sklearn import svm\n",
    "\n",
    "### Same trained models ###\n",
    "import h5py\n",
    "from keras.models import model_from_json\n",
    "import pickle"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# III. Import datas"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-03-06T07:44:28.408712Z",
     "start_time": "2019-03-06T07:44:28.403615Z"
    }
   },
   "outputs": [],
   "source": [
    "path = '/Users/maelfabien/filrouge_pole_emploi/Video/'\n",
    "local_path = '/Users/maelfabien/Desktop/LocalDB/Videos/'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-03-06T07:45:19.317829Z",
     "start_time": "2019-03-06T07:44:28.416302Z"
    },
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "pd.options.mode.chained_assignment = None  # default='warn'  #to suppress SettingWithCopyWarning\n",
    "\n",
    "#Reading the dataset\n",
    "dataset = pd.read_csv(local_path + 'fer2013.csv')\n",
    "\n",
    "#Obtaining train data where usage is \"Training\"\n",
    "train = dataset[dataset[\"Usage\"] == \"Training\"]\n",
    "\n",
    "#Obtaining test data where usage is \"PublicTest\"\n",
    "test = dataset[dataset[\"Usage\"] == \"PublicTest\"]\n",
    "\n",
    "#Converting \" \" separated pixel values to list\n",
    "train['pixels'] = train['pixels'].apply(lambda image_px : np.fromstring(image_px, sep = ' '))\n",
    "test['pixels'] = test['pixels'].apply(lambda image_px : np.fromstring(image_px, sep = ' '))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-03-06T07:45:19.364177Z",
     "start_time": "2019-03-06T07:45:19.335768Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>emotion</th>\n",
       "      <th>pixels</th>\n",
       "      <th>Usage</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0</td>\n",
       "      <td>70 80 82 72 58 58 60 63 54 58 60 48 89 115 121...</td>\n",
       "      <td>Training</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>0</td>\n",
       "      <td>151 150 147 155 148 133 111 140 170 174 182 15...</td>\n",
       "      <td>Training</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2</td>\n",
       "      <td>231 212 156 164 174 138 161 173 182 200 106 38...</td>\n",
       "      <td>Training</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>4</td>\n",
       "      <td>24 32 36 30 32 23 19 20 30 41 21 22 32 34 21 1...</td>\n",
       "      <td>Training</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>6</td>\n",
       "      <td>4 0 0 0 0 0 0 0 0 0 0 0 3 15 23 28 48 50 58 84...</td>\n",
       "      <td>Training</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   emotion                                             pixels     Usage\n",
       "0        0  70 80 82 72 58 58 60 63 54 58 60 48 89 115 121...  Training\n",
       "1        0  151 150 147 155 148 133 111 140 170 174 182 15...  Training\n",
       "2        2  231 212 156 164 174 138 161 173 182 200 106 38...  Training\n",
       "3        4  24 32 36 30 32 23 19 20 30 41 21 22 32 34 21 1...  Training\n",
       "4        6  4 0 0 0 0 0 0 0 0 0 0 0 3 15 23 28 48 50 58 84...  Training"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dataset.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-03-06T07:45:19.400868Z",
     "start_time": "2019-03-06T07:45:19.371931Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>emotion</th>\n",
       "      <th>pixels</th>\n",
       "      <th>Usage</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>299</th>\n",
       "      <td>1</td>\n",
       "      <td>126 126 129 120 110 168 174 172 173 174 170 15...</td>\n",
       "      <td>Training</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>388</th>\n",
       "      <td>1</td>\n",
       "      <td>89 55 24 40 43 48 53 55 59 41 33 31 22 32 42 4...</td>\n",
       "      <td>Training</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>416</th>\n",
       "      <td>1</td>\n",
       "      <td>204 195 181 131 50 50 57 56 66 98 138 161 173 ...</td>\n",
       "      <td>Training</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>473</th>\n",
       "      <td>1</td>\n",
       "      <td>14 11 13 12 41 95 113 112 111 122 132 137 142 ...</td>\n",
       "      <td>Training</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>533</th>\n",
       "      <td>1</td>\n",
       "      <td>18 25 49 75 89 97 100 100 101 103 105 107 107 ...</td>\n",
       "      <td>Training</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "     emotion                                             pixels     Usage\n",
       "299        1  126 126 129 120 110 168 174 172 173 174 170 15...  Training\n",
       "388        1  89 55 24 40 43 48 53 55 59 41 33 31 22 32 42 4...  Training\n",
       "416        1  204 195 181 131 50 50 57 56 66 98 138 161 173 ...  Training\n",
       "473        1  14 11 13 12 41 95 113 112 111 122 132 137 142 ...  Training\n",
       "533        1  18 25 49 75 89 97 100 100 101 103 105 107 107 ...  Training"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dataset[dataset['emotion'] == 1].head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-03-06T07:45:20.102014Z",
     "start_time": "2019-03-06T07:45:19.406868Z"
    }
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAs8AAAF1CAYAAAAXywc5AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4wLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvqOYd8AAAHrBJREFUeJzt3Xm0pWddJ/rvj1QiBJAp1TQZKxdobgP3ttCRobFtWhQCQcPqi4oLMdLRrLZpp8aWgHYnQmjjvV5B71J6pQkzEuigJgItRIZGVIaEQQyDZIVABiAFISGADIHf/eN9Ku46nEo9lZyqc5J8PmvtVfudnv17h13nu9/9vO+u7g4AALB3d9jsAgAA4NZCeAYAgEnCMwAATBKeAQBgkvAMAACThGcAAJgkPMOtQFX9t6r6zxvU1tFV9eWqOmgMv6OqfmYj2h7t/c+qOmmj2tuH1z2jqj5fVZ+dnP/0qnrV/q5ro1XVy6rqjE167aqql1bVF6vqvetMf2pVvWUzauOmVdVzqurFm10H3BZs2+wC4Pauqi5Lcu8kNyT5VpKPJHlFkrO6+9tJ0t3/bh/a+pnu/vM9zdPdn05yl1tW9Y2vd3qS+3X3T660//iNaHsf6zg6yTOTHNPdV68z/dFJXtXdRx7o2m5jvi/JDyU5sru/snZid786yasPeFXsZr3jvbv/6+ZVBLctzjzD1vDD3X3XJMckOTPJs5KcvdEvUlW31Q/MRyf5wnrBmT3b9e3DPjgmyWXrBWd2dxt+r8HtnvAMW0h3X9fd5yf58SQnVdWDk92/qq+qw6rqDVV1bVVdU1V/UVV3qKpXZgmRfzq6ZfxqVe2oqq6qk6vq00netjJu9Y/7favqvVX1pao6r6ruOV7r0VV1xWqNVXVZVf1gVR2f5DlJfny83ofG9Bu7gYy6fr2qPlVVV1fVK6rqbmParjpOqqpPjy4Xv7anbVNVdxvL7xzt/fpo/weTXJDk8FHHy9Ysd+ck/3Nl+per6vAx+ZDR5vVVdXFVHbey3OFV9frxep+sql+4idpeVlW/X1VvHG29p6ruu2Y9t63Mv7qNfrqq/rKqXjD26aVV9S/G+MvHdlvbDeawqrpgvNb/qqpjVtr+38e0a6rq41X1Y2vqfFFVvamqvpLkX6+zLodX1flj+Uuq6mfH+JOTvDjJI8c2/I11lv3pqnrXynBX1b+vqk+MWp9XVfetqr8ax9rrquqQMe89xnG9s5ZuIW+oqiNX2jq2qt452vnzsb1ftTL9EaPda6vqQ7WcfV2t69Kx7Cer6ql72I+nV9W5VfXaMe/7q+qfrdk26x4TK8u+qqq+lOSn12n/u6rqt8fx/rlaumPdaUx7dFVdUcv79uqq+kxVPamqnlBVfzf2x3PWtPXCqrpqPF44xq17vNeabkpV9SO1HPPXjuPxn65Mu6yqfqWq/qaqrhvb447rbTO4XepuDw+PTXwkuSzJD64z/tNJfm48f1mSM8bz30zy35IcPB7/Mkmt11aSHUk6SzeQOye508q4bWOedyS5MsmDxzyvz/KVb5I8OskVe6o3yem75l2Z/o4sXUeS5N8muSTJ/5alq8gfJXnlmtr++6jrnyX5epJ/uoft9Iok5yW561j275KcvKc61yy73nqcnuRrSZ6Q5KCxXd89pt0hyUVJ/kuSQ0b9lyZ53B7af1mSLyR5WJbucK9Ocs6a9dy2h23001m67Dx91HHG2Pe/n+S7kjw2yfVJ7rLyWtcn+f4x/XeTvGtMu3OSy0db25I8JMnnkzxwZdnrkjxqrOMd11mXdyb5gyR3TPI9SXYm+YGVWt91E9t5t+ljvc9L8t1JHjT271vH9rxbli5KJ41575Xk/0py6NjH/yPJn6y09ddJfnvsj+9L8qX8w3F6xNj+Txjr9UNjePvYJl9K8oAx732SPGgP9Z+e5JtJnpzlvfUrST45nt/kMbGy7JPGvHdap/0XJDk/yT3HOv5pkt9cOUZvGO0fnORnx7b/wzHvg5L8fZJjx/zPTfLuJP9orOdfJXneXo73XdvrnyT5ythOByf51Szv00NW3uPvTXL4qPWjSf7dZv9f6eGxVR7OPMPWdVWWP1xrfTNLADimu7/Z3X/R3b2Xtk7v7q9099/vYforu/tve/k6/j8n+bHa96/01/PUJL/T3Zd295eTPDvJU2r3s96/0d1/390fSvKhLCF6N6OWpyR5dndf392XJfl/kzztFtb3ru5+U3d/K8krV177e5Ns7+7ndvc3uvvSLCH/KTfR1h9393u7+4Ys4fl79qGOT3b3S0cdr01yVJLndvfXu/stSb6R5H4r87+xu9/Z3V9P8mtZzgYfleSJWbpVvLS7b+juD2T5MPSjK8ue191/2d3f7u6vrRYx2nhUkmd199e6+4NZzjb/1D6sy1r/d3d/qbsvTvK3Sd4yjofrspwhfUiSdPcXuvv13f3V7r4+yfOT/KtR19FZ9sl/GfvjXVlC6C4/meRNY19+u7svSHJhljCdJN9O8uCqulN3f2bUsicXdfe53f3NJL+T5UPEIzJ3TPx1d//JqGG391pVVZJTkvxyd18z1vG/rln+m0meP177nCSHJfndccxfnOXDxq5j9KlZjpGru3tnkt/I/Pvhx7McQxeM1/rtLB9g/8XKPL/X3Vd19zVZQv6+HM9wm6ZPFmxdRyS5Zp3x/0+Ws0hvWf4e56zuPnMvbV2+D9M/leVs1GFzZd6kw0d7q21vy3KB5C6rd8f4ata/mPGwUdPato64hfWtfe07jmB/TJavva9dmX5Qkr/Yh7b25aLMz608//sk6e6141bbu3F/dfeXq+qaLNv6mCQPX1P3tiwfDL5j2XUcnmRXsNvlU0mO28P8M9aux9rhf5wkVXVoljOzxye5x5h+1/HBaVddX11Z9vIsHzKSZb1/tKp+eGX6wUne3t1fqaofz3IW+eyq+sskz+zuj+2h3tVt++1aui0dnuUs+t6OiZvattuznFW/aLxvk6RGG7t8YXyASsZxkO/cXruOg/XeW4dnzm7LjvW8PLu/n9Yez7Ntw22e8AxbUFV9b5Y/ZO9aO20Em2cmeWYtfaLfVlXv6+63ZvkDv569nZk+auX50VnOgH0+y1e7h67UdVCWEDDb7lVZgs1q2zdkCQT7cueLz4+ajsly9m1XW1dOLr+3Ote6PMvZ4Pvv43Lr2XVx3aFZug8kIzDeAjfur6q6S5ZvKK7KUvf/6u4fuollb2pbXJXknlV115UAvS/b+ZZ4ZpIHJHl4d3+2qr4nyQeyBMzPjLoOXQnQq8fs5Vm+PfnZ9Rru7jcnefPoX3xGljPG/3IPdaxu2ztkOU6vynLc7u2YuKlt+/ks4fdB3b0R23PXe2vXWfSjx7i91bFr2f9j18A4K35UDsx+hls93TZgC6mq766qJ2b5yvZV3f3hdeZ5YlXdb/zBuy7L7e2+PSZ/LktfzH31k1X1wHH277lJzh1nwP4uy9nYE6rq4CS/nqWf7S6fS7JjhIz1vCbJL4+Lve6S5Wvq146uDdNGLa9L8vyqumstF8j9xySz92n+XJJ71bhYccJ7k1xfVc+qqjtV1UFV9eDxoWafjK/Ur8yyjQ+qqn+b5L772s4aT6iq76vlYrvnZemrfXmSNyT5J1X1tKo6eDy+d/VisL3UenmWvrO/WVV3rKr/M8nJmd/Ot8Rds4TLa2u5YPW0lbo+laUbxulVdUhVPTLJ6lnmVyX54ap63NjGdxwX4B1ZVfeuqhPHhXRfT/Ll/MP7ZT3/vKr+zfgG4pfGMu/OLTwmernt5H9P8oKq+kdJUlVHVNXjprbOd3pNkl+vqu1VdViWvtK79tPejvfXJTmhqh4z3tfPHOv5VzezFrhdEZ5ha/jTqro+yxm0X8vS1/Lpe5j3/kn+PEsI+Oskf9Ddbx/TfjPLH9Rrq+pX9uH1X5nlYrLPZunj+QvJcvePJP8+S7/XK7OcRV29+8b/GP9+oarev067LxltvzPLhVdfS/Lz+1DXqp8fr39pljPyfzja36vxFf1rklw6ts1NfgU9wvoTs/Tz/GSWs4YvznKR283xs0n+U5aL2B6UWx5S/jBLuLwmyT/P0ud317cSj83Sj/aqLPvzt7L7B569+YksFzleleSPk5zWN3Hf8A30wiz9bj+fJaz+2ZrpT03yyCzb8IwsfcO/ntwY+k/McveXnVneR/8py9+4O2T5oHVVlu31r5L83E3UcV6WPsFfzNKH+N+Maws24ph4VpYL895dyx05/jzL2fab44wsHyj+JsmHk7x/jNvr8d7dH89yzPx/Yz1+OMvtMr9xM2uB25VdV+gDwK1GVb02yce6+7S9zjzf5ulZ86M/AGs58wzAlje6n9y3lnt7H5/lTPOfbHZdwO2PCwYBuDX4x1nuE36vLF2Hfm7cig/ggNJtAwAAJum2AQAAk4RnAACYtKX7PB922GG9Y8eOzS4DAIDbuIsuuujz3b19b/Nt6fC8Y8eOXHjhhZtdBgAAt3FV9am9z6XbBgAATBOeAQBgkvAMAACThGcAAJgkPAMAwCThGQAAJgnPAAAwSXgGAIBJwjMAAEwSngEAYJLwDAAAk4RnAACYJDwDAMCkbZtdAAAba8epb9zQ9i4784QNbQ/g1syZZwAAmCQ8AwDAJOEZAAAmCc8AADBJeAYAgEnCMwAATBKeAQBgkvAMAACThGcAAJgkPAMAwCThGQAAJgnPAAAwSXgGAIBJwjMAAEwSngEAYJLwDAAAk4RnAACYJDwDAMAk4RkAACYJzwAAMEl4BgCAScIzAABMEp4BAGCS8AwAAJOEZwAAmCQ8AwDAJOEZAAAmCc8AADBJeAYAgEnCMwAATJoKz1X1y1V1cVX9bVW9pqruWFXHVtV7quqSqnptVR0y5v2uMXzJmL5jpZ1nj/Efr6rH7Z9VAgCA/WOv4bmqjkjyC0mO6+4HJzkoyVOS/FaSF3T3/ZJ8McnJY5GTk3xxjH/BmC9V9cCx3IOSHJ/kD6rqoI1dHQAA2H9mu21sS3KnqtqW5NAkn0nyA0nOHdNfnuRJ4/mJYzhj+mOqqsb4c7r76939ySSXJHnYLV8FAAA4MPYanrv7yiS/neTTWULzdUkuSnJtd98wZrsiyRHj+RFJLh/L3jDmv9fq+HWWuVFVnVJVF1bVhTt37rw56wQAAPvFTLeNe2Q5a3xsksOT3DlLt4v9orvP6u7juvu47du376+XAQCAfTbTbeMHk3yyu3d29zeT/FGSRyW5++jGkSRHJrlyPL8yyVFJMqbfLckXVsevswwAAGx5M+H500keUVWHjr7Lj0nykSRvT/LkMc9JSc4bz88fwxnT39bdPcY/ZdyN49gk90/y3o1ZDQAA2P+27W2G7n5PVZ2b5P1JbkjygSRnJXljknOq6owx7uyxyNlJXllVlyS5JssdNtLdF1fV67IE7xuSPKO7v7XB6wMAAPvNXsNzknT3aUlOWzP60qxzt4zu/lqSH91DO89P8vx9rBEAALYEvzAIAACThGcAAJgkPAMAwCThGQAAJgnPAAAwSXgGAIBJwjMAAEwSngEAYJLwDAAAk4RnAACYJDwDAMAk4RkAACYJzwAAMEl4BgCAScIzAABMEp4BAGCS8AwAAJOEZwAAmCQ8AwDAJOEZAAAmCc8AADBJeAYAgEnCMwAATBKeAQBgkvAMAACThGcAAJgkPAMAwCThGQAAJgnPAAAwSXgGAIBJwjMAAEwSngEAYJLwDAAAk4RnAACYJDwDAMAk4RkAACYJzwAAMEl4BgCAScIzAABMEp4BAGCS8AwAAJOEZwAAmCQ8AwDAJOEZAAAmCc8AADBJeAYAgEnCMwAATBKeAQBgkvAMAACThGcAAJgkPAMAwCThGQAAJgnPAAAwSXgGAIBJwjMAAEwSngEAYJLwDAAAk7ZtdgEAANw67Tj1jRva3mVnnrCh7e0PzjwDAMAk4RkAACZNheequntVnVtVH6uqj1bVI6vqnlV1QVV9Yvx7jzFvVdXvVdUlVfU3VfXQlXZOGvN/oqpO2l8rBQAA+8Nsn+ffTfJn3f3kqjokyaFJnpPkrd19ZlWdmuTUJM9K8vgk9x+Phyd5UZKHV9U9k5yW5LgkneSiqjq/u7+4oWsEAFvQ7bFvKNwW7fXMc1XdLcn3Jzk7Sbr7G919bZITk7x8zPbyJE8az09M8opevDvJ3avqPkkel+SC7r5mBOYLkhy/oWsDAAD70Uy3jWOT7Ezy0qr6QFW9uKrunOTe3f2ZMc9nk9x7PD8iyeUry18xxu1p/G6q6pSqurCqLty5c+e+rQ0AAOxHM+F5W5KHJnlRdz8kyVeydNG4UXd3lq4Yt1h3n9Xdx3X3cdu3b9+IJgEAYEPMhOcrklzR3e8Zw+dmCdOfG90xMv69eky/MslRK8sfOcbtaTwAANwq7DU8d/dnk1xeVQ8Yox6T5CNJzk+y644ZJyU5bzw/P8lPjbtuPCLJdaN7x5uTPLaq7jHuzPHYMQ4AAG4VZu+28fNJXj3utHFpkqdnCd6vq6qTk3wqyY+Ned+U5AlJLkny1TFvuvuaqnpekveN+Z7b3ddsyFoAAMABMBWeu/uDWW4xt9Zj1pm3kzxjD+28JMlL9qVAAADYKvzCIAAATBKeAQBgkvAMAACThGcAAJgkPAMAwCThGQAAJgnPAAAwSXgGAIBJwjMAAEwSngEAYJLwDAAAk4RnAACYJDwDAMAk4RkAACYJzwAAMEl4BgCAScIzAABMEp4BAGCS8AwAAJOEZwAAmCQ8AwDAJOEZAAAmCc8AADBJeAYAgEnCMwAATNq22QUAt247Tn3jhrZ32ZknbGh7ALCRnHkGAIBJwjMAAEwSngEAYJLwDAAAk4RnAACY5G4be+AOAgAArOXMMwAATBKeAQBgkvAMAACThGcAAJgkPAMAwCThGQAAJgnPAAAwSXgGAIBJwjMAAEwSngEAYJLwDAAAk4RnAACYJDwDAMAk4RkAACYJzwAAMEl4BgCAScIzAABMEp4BAGCS8AwAAJOEZwAAmCQ8AwDAJOEZAAAmCc8AADBJeAYAgEnCMwAATBKeAQBgkvAMAACThGcAAJg0HZ6r6qCq+kBVvWEMH1tV76mqS6rqtVV1yBj/XWP4kjF9x0obzx7jP15Vj9volQEAgP1pX848/2KSj64M/1aSF3T3/ZJ8McnJY/zJSb44xr9gzJeqemCSpyR5UJLjk/xBVR10y8oHAIADZyo8V9WRSU5I8uIxXEl+IMm5Y5aXJ3nSeH7iGM6Y/pgx/4lJzunur3f3J5NckuRhG7ESAABwIMyeeX5hkl9N8u0xfK8k13b3DWP4iiRHjOdHJLk8Scb068b8N45fZxkAANjytu1thqp6YpKru/uiqnr0/i6oqk5JckqSHH300fv75QCA26kdp75xQ9u77MwTNrQ9tqaZM8+PSvIjVXVZknOydNf43SR3r6pd4fvIJFeO51cmOSpJxvS7JfnC6vh1lrlRd5/V3cd193Hbt2/f5xUCAID9Za/hubuf3d1HdveOLBf8va27n5rk7UmePGY7Kcl54/n5Yzhj+tu6u8f4p4y7cRyb5P5J3rthawIAAPvZXrtt3IRnJTmnqs5I8oEkZ4/xZyd5ZVVdkuSaLIE73X1xVb0uyUeS3JDkGd39rVvw+gAAcEDtU3ju7nckecd4fmnWuVtGd38tyY/uYfnnJ3n+vhYJAABbgV8YBACAScIzAABMEp4BAGCS8AwAAJOEZwAAmCQ8AwDAJOEZAAAmCc8AADBJeAYAgEnCMwAATBKeAQBgkvAMAACThGcAAJgkPAMAwCThGQAAJgnPAAAwSXgGAIBJwjMAAEwSngEAYJLwDAAAk4RnAACYJDwDAMAk4RkAACYJzwAAMEl4BgCAScIzAABMEp4BAGCS8AwAAJOEZwAAmCQ8AwDAJOEZAAAmCc8AADBJeAYAgEnCMwAATBKeAQBgkvAMAACThGcAAJgkPAMAwCThGQAAJgnPAAAwSXgGAIBJwjMAAEwSngEAYJLwDAAAk4RnAACYJDwDAMAk4RkAACYJzwAAMEl4BgCAScIzAABMEp4BAGCS8AwAAJOEZwAAmCQ8AwDAJOEZAAAmCc8AADBJeAYAgEnCMwAATBKeAQBgkvAMAACT9hqeq+qoqnp7VX2kqi6uql8c4+9ZVRdU1SfGv/cY46uqfq+qLqmqv6mqh660ddKY/xNVddL+Wy0AANh4M2eeb0jyzO5+YJJHJHlGVT0wyalJ3trd90/y1jGcJI9Pcv/xOCXJi5IlbCc5LcnDkzwsyWm7AjcAANwa7DU8d/dnuvv94/n1ST6a5IgkJyZ5+Zjt5UmeNJ6fmOQVvXh3krtX1X2SPC7JBd19TXd/MckFSY7f0LUBAID9aJ/6PFfVjiQPSfKeJPfu7s+MSZ9Ncu/x/Igkl68sdsUYt6fxAABwqzAdnqvqLklen+SXuvtLq9O6u5P0RhRUVadU1YVVdeHOnTs3okkAANgQU+G5qg7OEpxf3d1/NEZ/bnTHyPj36jH+yiRHrSx+5Bi3p/G76e6zuvu47j5u+/bt+7IuAACwX83cbaOSnJ3ko939OyuTzk+y644ZJyU5b2X8T427bjwiyXWje8ebkzy2qu4xLhR87BgHAAC3Ctsm5nlUkqcl+XBVfXCMe06SM5O8rqpOTvKpJD82pr0pyROSXJLkq0meniTdfU1VPS/J+8Z8z+3uazZkLQAA4ADYa3ju7nclqT1Mfsw683eSZ+yhrZckecm+FAgAAFuFXxgEAIBJwjMAAEwSngEAYJLwDAAAk4RnAACYJDwDAMAk4RkAACYJzwAAMEl4BgCAScIzAABMEp4BAGCS8AwAAJOEZwAAmCQ8AwDAJOEZAAAmCc8AADBJeAYAgEnCMwAATBKeAQBgkvAMAACThGcAAJgkPAMAwCThGQAAJgnPAAAwSXgGAIBJwjMAAEwSngEAYJLwDAAAk4RnAACYJDwDAMAk4RkAACYJzwAAMEl4BgCAScIzAABMEp4BAGCS8AwAAJOEZwAAmCQ8AwDAJOEZAAAmCc8AADBJeAYAgEnCMwAATBKeAQBgkvAMAACThGcAAJi0bbMLgFk7Tn3jhrd52ZknbHibAMBtlzPPAAAwSXgGAIBJwjMAAEwSngEAYJLwDAAAk4RnAACYJDwDAMAk4RkAACYJzwAAMEl4BgCAScIzAABMEp4BAGCS8AwAAJOEZwAAmHTAw3NVHV9VH6+qS6rq1AP9+gAAcHMd0PBcVQcl+f0kj0/ywCQ/UVUPPJA1AADAzXWgzzw/LMkl3X1pd38jyTlJTjzANQAAwM1yoMPzEUkuXxm+YowDAIAtr7r7wL1Y1ZOTHN/dPzOGn5bk4d39H1bmOSXJKWPwAUk+fsAK3N1hST6/Sa/N+uyTrcl+2Xrsk63Jftl67JOtZzP3yTHdvX1vM207EJWsuDLJUSvDR45xN+rus5KcdSCLWk9VXdjdx212HfwD+2Rrsl+2Hvtka7Jfth77ZOu5NeyTA91t431J7l9Vx1bVIUmekuT8A1wDAADcLAf0zHN331BV/yHJm5MclOQl3X3xgawBAABurgPdbSPd/aYkbzrQr3szbHrXEb6DfbI12S9bj32yNdkvW499svVs+X1yQC8YBACAWzM/zw0AAJOE5zX8fPjWU1Uvqaqrq+pvN7sWFlV1VFW9vao+UlUXV9UvbnZNJFV1x6p6b1V9aOyX39jsmlhU1UFV9YGqesNm18Kiqi6rqg9X1Qer6sLNroekqu5eVedW1ceq6qNV9cjNrmk9um2sGD8f/ndJfijLD7i8L8lPdPdHNrWw27mq+v4kX07yiu5+8GbXQ1JV90lyn+5+f1XdNclFSZ7kvbK5qqqS3Lm7v1xVByd5V5Jf7O53b3Jpt3tV9R+THJfku7v7iZtdD0t4TnJcd7vP8xZRVS9P8hfd/eJxV7ZDu/vaza5rLWeed+fnw7eg7n5nkms2uw7+QXd/prvfP55fn+Sj8Wuhm64XXx6DB4+HMySbrKqOTHJCkhdvdi2wVVXV3ZJ8f5Kzk6S7v7EVg3MiPK/l58NhH1XVjiQPSfKeza2E5MbuAR9McnWSC7rbftl8L0zyq0m+vdmFsJtO8paqumj8ujGb69gkO5O8dHRxenFV3Xmzi1qP8AzcbFV1lySvT/JL3f2lza6HpLu/1d3fk+UXXB9WVbo6baKqemKSq7v7os2uhe/wfd390CSPT/KM0UWQzbMtyUOTvKi7H5LkK0m25LVnwvPu9vrz4cBi9Kl9fZJXd/cfbXY97G583fn2JMdvdi23c49K8iOjf+05SX6gql61uSWRJN195fj36iR/nKXrJpvniiRXrHxbdm6WML3lCM+78/PhMGFcmHZ2ko929+9sdj0sqmp7Vd19PL9TloufP7a5Vd2+dfezu/vI7t6R5W/K27r7Jze5rNu9qrrzuNg5o2vAY5O4o9Mm6u7PJrm8qh4wRj0myZa8CP2A/8LgVubnw7emqnpNkkcnOayqrkhyWnefvblV3e49KsnTknx49K9NkueMXxBl89wnycvHnYPukOR13e3WaPCd7p3kj5fzANmW5A+7+882tySS/HySV48TmJcmefom17Mut6oDAIBJum0AAMAk4RkAACYJzwAAMEl4BgCAScIzAABMEp4BAGCS8AwAAJOEZwAAmPT/AyqCv+ZJGmSLAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 864x432 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.figure(figsize=(12,6))\n",
    "plt.hist(dataset['emotion'], bins=30)\n",
    "plt.title(\"Distribution of the number of images per emotion\")\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-03-06T07:45:20.120068Z",
     "start_time": "2019-03-06T07:45:20.108576Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(28709, 3)"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "train.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-03-06T07:45:20.143203Z",
     "start_time": "2019-03-06T07:45:20.132550Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(3589, 3)"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "test.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# IV. Create the data set"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-03-06T07:45:20.152641Z",
     "start_time": "2019-03-06T07:45:20.146615Z"
    }
   },
   "outputs": [],
   "source": [
    "shape_x = 48\n",
    "shape_y = 48"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-03-06T07:45:21.167952Z",
     "start_time": "2019-03-06T07:45:20.173776Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Shape of X_train and y_train is (28709, 48, 48, 1) and (28709, 1) respectively.\n",
      "Shape of X_test and y_test is (3589, 48, 48, 1) and (3589, 1) respectively.\n"
     ]
    }
   ],
   "source": [
    "X_train = train.iloc[:, 1].values\n",
    "y_train = train.iloc[:, 0].values\n",
    "X_test = test.iloc[:, 1].values\n",
    "y_test = test.iloc[:, 0].values\n",
    "\n",
    "#np.vstack stack arrays in sequence vertically (picking element row wise)\n",
    "X_train = np.vstack(X_train)\n",
    "X_test = np.vstack(X_test)\n",
    "\n",
    "#Reshape X_train, y_train,X_test,y_test in desired formats\n",
    "X_train = np.reshape(X_train, (X_train.shape[0],48,48,1))\n",
    "y_train = np.reshape(y_train, (y_train.shape[0],1))\n",
    "X_test = np.reshape(X_test, (X_test.shape[0],48,48,1))\n",
    "y_test = np.reshape(y_test, (y_test.shape[0],1))\n",
    "\n",
    "print(\"Shape of X_train and y_train is \" + str(X_train.shape) +\" and \" + str(y_train.shape) +\" respectively.\")\n",
    "print(\"Shape of X_test and y_test is \" + str(X_test.shape) +\" and \" + str(y_test.shape) +\" respectively.\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-03-06T07:45:21.716077Z",
     "start_time": "2019-03-06T07:45:21.174714Z"
    }
   },
   "outputs": [],
   "source": [
    "# Change to float datatype\n",
    "train_data = X_train.astype('float32')\n",
    "test_data = X_test.astype('float32')\n",
    "\n",
    "# Scale the data to lie between 0 to 1\n",
    "train_data /= 255\n",
    "test_data /= 255\n",
    "\n",
    "# Change the labels from integer to categorical data\n",
    "train_labels_one_hot = to_categorical(y_train)\n",
    "test_labels_one_hot = to_categorical(y_test)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# V. Define the number of classes"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-03-06T07:45:21.743276Z",
     "start_time": "2019-03-06T07:45:21.727145Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Total number of outputs :  7\n",
      "Output classes :  [0 1 2 3 4 5 6]\n"
     ]
    }
   ],
   "source": [
    "# Find the unique numbers from the train labels\n",
    "classes = np.unique(y_train)\n",
    "nClasses = len(classes)\n",
    "print('Total number of outputs : ', nClasses)\n",
    "print('Output classes : ', classes)\n",
    "\n",
    "# Find the shape of input images and create the variable input_shape\n",
    "nRows,nCols,nDims = X_train.shape[1:]\n",
    "input_shape = (nRows, nCols, nDims)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-03-06T07:45:21.768160Z",
     "start_time": "2019-03-06T07:45:21.749378Z"
    }
   },
   "outputs": [],
   "source": [
    "#Defining labels \n",
    "def get_label(argument):\n",
    "    labels = {0:'Angry', 1:'Disgust', 2:'Fear', 3:'Happy', 4:'Sad' , 5:'Surprise', 6:'Neutral'}\n",
    "    return(labels.get(argument, \"Invalid emotion\"))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-03-06T07:45:22.505305Z",
     "start_time": "2019-03-06T07:45:21.780010Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Text(0.5, 1.0, 'Ground Truth : Surprise')"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAlMAAAEtCAYAAAAsgeXEAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4wLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvqOYd8AAAIABJREFUeJztvXm0XWd55vm8kmVs8KBZ1ix5HsJgxwFTZCWM1ZgMpKpYCUMI6Yb2SpNUU110JyRVXUUqE1SvqiRVXXEXKShMJQZSCRQ0pLvNuABjRGyBAVvEsjVYsmZZkuUBbEtf/3GOwtX5fq/17bvvPdK9PL+1tKT7au/9ze/57j7P+71RSpExxhhjjJkcc053BYwxxhhjZjLeTBljjDHG9MCbKWOMMcaYHngzZYwxxhjTA2+mjDHGGGN64M2UMcYYY0wPvJkyJxER2yLilaex/J0R8dLTVb4x5szGPmpqiYjbIuJNp7seMx1vpsZMRLw+IjZExGMRsW/477dHRJzuuj0TEfH/RMSjwz9PRcSTE37+vyb5zD+LiHdPcVWpnLdFRImIfzTdZRkz07GPOumZ0+qjImJBRHwwIvZExCMR8bcR8b9NV3lEKeXvl1L+fJxlzka8mRojEfFOSX8s6f+QdJGkZZJ+RdJLJJ2d3DN3bBV8BkopN5ZSziulnCfpzyX96xM/l1J+ZfT6iDhr/LVMeYukhyX90nQVcIa115hJYR81dv6dBv16paT5kn5O0gOTeVDX9kTEnIjwHmCKcEeOiYi4UNK/kvT2UspfllKOlgHfKKW8qZTy/eF1H4yImyPiryPiMUkvi4gLI+JDEbE/IrZHxD8/sQgi4t0R8WcTylk3fAtz1vDnL0bE70TE7RFxdPhKd/GE6988fObBiPhnPdr3yuHr99+KiD2S/nT4RuiLE645a1i3dRHxdkm/IOm3hr85fnzC466LiG9HxJGI+HBEPKtHvS7R4IPgJkk3RsQSqPOvD/t2V0T80oT/XxIRnx7+xvj1iPj9E+2Z0Ja3R8T9kr4bEf8xIt47Uv5fR8Q/nmz9jRkX9lGnxUf9mKRbSymHSynHSymbSikfG9bl0og4KUVJRHwlIn55+O+3RcSXIuLfRcTDkv75BNufDOu2KSJeNnL/70TEHZIek7Rm5JmXD+8/EhEHIuLWCfdeHRGfjYiHI+K74Tf9J+HN1Ph4saRnSfpEw7VvlPR7ks6X9BVJ/17ShZIulvSTGrxh+e87lP3G4fVLNfgt6H+VBotD0s2S3ixphaRFklZ1eO4oqySdJ2mNpLc/04WllD+R9FFJvz/8zfEfTPjvn5f0Kg3a+6PD+lVExPqIOBwRK56hqF+S9LVSyl9p8BvfG6HO52rQ/l+RdHNEXDD8v5slHdbgt/P/QYM3XKP8rAYO8bmSbpH0hojB1yERsUzSSyV9+BnqZ8yZgn3UBMbko74m6Q8i4pcj4rJuTZEk/T1JmyQtkfTeCbbvSlos6XckfSwi5k+4580a+LMLJO0ced7vSfq0pAUa9NV/GLbjPEmfkfQhDcboTZLeFxFXTKLOsxJvpsbHYkkHSilPnzBExFeHC+2JiPiJCdd+opRyeynluKSnJL1e0m8Of1PcJunfKFm8Cf+5lHJfKeUJSX8h6QVD++skfaqU8qXhb53/u6Tjk26h9LSkd5dSnhyWNVn+qJSyp5RyUNKnJtT3JEopW0sp80spu+j/h5uaX5J04rerW1V/1fc9Sb9bSnmqlPJJSd+XdHlEzNPglfu/KKU8UUr5jqT/AsX8finl0PCarw6f99Lh/71B0mdLKQca223M6cQ+qp0p8VEabOg+Kul/lrQpIjZHxN/vUI8HSyk3l1KOTWjPbkn/fujTbpW0VdKNE+75wPAN2FMTx3rIU5LWSVpeSvleKeX2of21ku4rpXyolPJ0KeUuSf9Ng/Ex8mZqnByUtDgmfK9dSvl7pZT5w/+bOBY7Jvx7saR5krZPsG2XtLJD2Xsm/PtxDX4zkwa/6f1dWaWUx4Z1mSx7SylP9rj/BFl9u/ITGvx29dHhz7dq8Hr+RyZcc6CUcgzKWyZprk4ei4n/zmwfkvSLw3//ongDZsyZiH1UO1Pio0opj5dSfreUcp0Gb90+Jumvhl+5tkA+aWcpZeLXg9s16MdnuucE79RgLO8cfo154m38WkkvGW6sD0fEYQ2+Al3eWM9ZjzdT4+MODd56vLbh2okL4YAGvy2snWBbI+mh4b8fk/TsCf93UYc67Za0+sQPEfFsDRb0ZCkjP5+qbqPXTzVv0WCOf3uokbh9WCZ9XTfKXg1+A574lcJquG60Df9F0j+IiGslXSLp/+5aaWNOE/ZR4/dRPyiolCOS/kCDjdk6Dep2os0naKnf6NegayRNfDOWtqmUsruU8rZSynJJv6rBV3nrNdiAfW74lu3En/NKKb/W0LQfCryZGhOllMOSflvSn0TE6yLi/BhEU7xA0nOe4b5jGrz2/r3hPWsl/VNJJwSd35T0ExGxZvjbzG92qNZfSvrpiPjxiDhbA/HpVM6JuyU9LyKeGxHnSvqXI/+/VwPNwZQzdECvk/RWDV7Bn/jzv0h6U5wiAqmU8pQGr7F/OyLOjYhr9IM3Ts9033YN2n2LpP9aSvler4YYMybso8broyQpIv5lRFwfEWdHxDkafN33sKTNGrz92iPpFyNibkTcpJM3rBnLI+LXhmL612vwS93/21ifn4+IE28UD2uw8Tom6ZOSromIN0bEvOGfF1oz9QO8mRojpZR/rYGT+XUNFuleSf9R0m9I+uoz3PqPNfgtZYsGYs9bJX1g+MzPaPA11rck3aXB9/et9blHg98+btXgN8BDqgWJk6aUcq+k35f0RUl/K+lLI5f8J0nPj4hDEfGXXZ8fERfHIMqGxJ3/UNJRSX821DbsKaXskfSnGgjOX9VQxP+kwW/BeyX9Zw2E5N9vuO8WDQTp/orPzCjso8bqo05wiwZfXe7SQG/5U8Ov/4qk/1HSb2nw9u9SSRsaiv2qpGs02JS9W9I/KqUcaqzyiyT9TQyiND8m6VdLKQ8O35r9dxr8Qrlbg03eH2gQsGAkxclfrRpjMiLi30iaX0p56ymue7mk90u6uHiBGWPGRES8TdIvllJeerrr8sOG30wZkzA8V+W5MeAGDUK3P36Ke86W9A5Jf+qNlDHG/HDgzZQxORdooJt6TIOv+N5TSkm/ooiI52rwNcRCDU42NsYY80OAv+YzxhhjjOmB30wZY4wxxvSg12YqIl4dgyzX90fEu6aqUsYYMw7sw4wxU8Gkv+YbntNznwYh5jsl/Y2kNwxDTZFFixaV1avp3MOa48cnnzEga9NTTz3VVM6cObzHnDu3PppomIbtlLYMqivZujyzC63j37dNfa6T+s2HLmR1ovb3/Yqc2nTs2LHKlpUzb968Jls2dq1zr/Vesu/Zs0eHDx+ensnbk64+bMGCBWX58pMPfM7m5dlnn13ZzjrrrMpG/Uh+RmL/9f3v1yd10ByQpMcee6zJ1urnJPaVZMvuJ3vmf/vcPx3+c7p8civj/JxopW/5XXwqXUvrsYufI/vWrVsPlFKWnKo+9epu54WS7i+lbJGkiPiIBifnppup1atX67bbbju5AuBgJOnxxx+vbK2dR05HGjj2UciZnH/++Xj/c55Tn1v3rGfVx2yQI82cLtmffLLOdpA5yNYFlZVPH95ENk4E1Z8c3NNPj6aFyutEHxpdNlitCypbZK317+IMaI4/+uijlY3aLknLli2rbCtX1hk8srlD9W/9hSNbY6PPfNvb3obXnSF08mHLly/XrbfeepLtkUcewQevW7eusi1cuLCy0RjMnz+/sknsvzZv3lzZVqzgI43uuOOOyrZx48bKRn7u3HPPxWc++9nPrmx0bbZBIl95zjnnVLZsg0lzu9Und1mr5P+yOrXS9xfELpup1g1ql5cLfTfSBPn+7DOq9ZcLstFnVGZ/85vfvB0urejzNd9KnZzjZ6e65WIyxpjTiX2YMWZKmHYBekTcFBF3RsSdBw/2yU9pjDHjZaL/Onz48OmujjHmDKXPZuohnZz4dZV+kNjy7yilvK+Ucn0p5fpFi/rkpzTGmCnllD5sov/Kvn4zxpg+mqm/kXTZMKP0Q5JeL+mNz3RDKaX6/jP73pi+D810GqOQFuVE+aPQ96nZd+lUV9I8ZOW31om+d87a3vq9fZfv56ks+t66y/fzXTRPVFbr9/MZ9F049X2m4yLNRKuAPNMstdY/00yRXueCCy5oLp/sNPZPPPFEZct0DKP9N67AgUnSyYcdO3ZMo2/XM+0FzSPqR9InZdD9tAazObxv377K1irY7eKnSbOU+SlaA9OxhlrbKfUPKmqFfEq2rqYj2KV1nLv4+ekIVOrLuMqZ9GaqlPJ0RPyapP9P0lxJHxgmpTTGmDMe+zBjzFTR582USil/Lemvp6guxhgzVuzDjDFTgU9AN8YYY4zpgTdTxhhjjDE98GbKGGOMMaYHvTRTk2FUWd8lGoYiHSjCJYumO3LkSFM5dFpuBkU/0Am+GdSm733ve5Uti+ZrPZk8ixJpPfGWommySBoqiyJ8utzf57TuDCqny0n1FCXSJRKp9aT6rE40T2iOZ+XTmNCJ1jSfKXOAVK+9MzyarxPHjh2r2p1FClEEJvkVOm6BxlWSjh49WtkoW0Pm/w4cOFDZWk/Rzvw0za0umQVofnTxv63PzCIcCWo/+ZrM97bO+S6pn4jW7BUSt7/vCeat7exygjrRd+zo/r4nxeO9k77TGGOMMcZ4M2WMMcYY0wdvpowxxhhjeuDNlDHGGGNMD8YqQC+lVEK+TERHAk6y0f1Z+g1KVEoC0PPOOw/vJxEmlUUC0kzwRoLfc889t+k6qV0EmAlAW1MKtKa9kVhYSaLYTChLfdUqliRRdfbM1vGUWMTYKt7v0veUdqOLeJcCMrIEvSRepnbSddkaGU2PkonfZyKllGrOZOuyNU0KjWEmQKe5SflOt27divc/+uijlY1E1DTfsnFsTdWR+YpWsXdGnzQvXQTcXUT1rSLuLuW3+uQuvqJV/N8lHUtrerPsua2+P7u2NW1Z9szWgC7Cb6aMMcYYY3rgzZQxxhhjTA+8mTLGGGOM6YE3U8YYY4wxPRi7AH1U4JqdbkqiMRKHkVhx7969+EwSmy9evLiyZWLLCy64oLK1CgtJGCyxCJvangmrWwXLfYWBdH8m4nvkkUearh0VKz9TWdSndCI02bJnktiyi2Ca6tTlBF26n+ZTdiI03U8nk5PwWGo/QZ4E0a3zqYvI9kzn2LFj1fzKBOg0j1pPC88E2GSnubF79268n8ZxwYIFTfXMhLmtQvuM1vWSPZP6hK6l+mfBO7QGWwN1JB6T1mwHmaC+iwicaPVrraJwqT34KbuOxo7KzwIyKLCGyqK2Z3XqEzDjN1PGGGOMMT3wZsoYY4wxpgfeTBljjDHG9MCbKWOMMcaYHoxVgB4RlXC1i+Du4YcfrmwPPvhgZSOhuCQtXLiwsu3Zs6eyHThwoPl+Oh2axI6ZgJPaTyI4EnVL0jnnnFPZSACZCe66nGzeWqfWU8AzYfSRI0cqG4kQ6bTbLqcnE5nQn8aZ+r6LUJSuJfFsJkCnfiKyE9BbTwumdmbzaXR9zyYBOp2Anp0E3xrcQAE42Sn8lBmB/Mq2bdvwflpvdII6BRdk/quP2FpqFzxn97eeDN4qlpbYf9E4ZcFTrXTp574nzdPco37ucgJ4a5aSLPiK5mOrT5LYV7ee6J+1s0ug1ih+M2WMMcYY0wNvpowxxhhjeuDNlDHGGGNMD7yZMsYYY4zpgTdTxhhjjDE9GHs6mVFlfhY5t2XLlspGqUKWLVtW2ShtTFYWRRRkEToUPbV///7KRhFl2TOXLl1a2SjFA0XySBzpMBpx9ExQpEVrKowsEpMiSg4ePFjZsrGncW5NL5Sl96AxoTZl0XxUf+pniiSliCmJ29SaYkbiNrWmzZGkQ4cOVTaKfKGUS9l8Hh27LlFUM4HR/umyLlvTD2WRW3QtzcHt27fj/fv27atsFKlJ403RrBKvoS4pZih6i+ZwFmVF7W+NRs6ua03R1WVu0zOp7lmUGfUzPTPzFURr1GKWoot8IqWzIpvEbSW/krWJ6kXzmdKWdYlabMVvpowxxhhjeuDNlDHGGGNMD7yZMsYYY4zpgTdTxhhjjDE9GKsA/emnn64E23v37sVrKSXCqlWrKhuJzTMBNtmXLFlS2TKxJQl+SYBOou5MBNwqICcRncSCQSo/E3CSEI/qRGJFEkVKnPqFBKgkFszsrWlJujyT6p+l8qDxo1REjz/+eGXLRMqt6RCyFDk0zjR2WXolup/EonQdiZSlWkDaRRB7pjNnzpxqLDNfQeuS5jAJbrPxpjlIQQRd5vCuXbsqGwnVKZVWZr/wwgsrWxYYQn6lSzouWsPUTvJ/mYCc+r9L6hW6lp7ZmkpH4na2+jSJ5x6ta5pPNB8kTu9G8z4LCKNAK5q7Wdoxaj/NPZpjXYJ6WvGbKWOMMcaYHngzZYwxxhjTA2+mjDHGGGN64M2UMcYYY0wPTrsAPRMmtgqzScRHIuBnKmuUTPBGdSIBKZVDwjyp/VTjTCxJdhIhZqJ8Eou3CkCzNrWe7JuJd0kETQLKvifbdjmBmLjooosqG/Un2SQeJyo/m880d+iU/uy0ciqL7qc5lolSR+c+iVxnKpTBITsxn+ZW6ynYmf+hQAY67Ty7P1tvo9DJ0rt378ZrDx8+XNloXmanjZOvXL58eWUjYbHEImQSoNM4ZWJtEiG3nhYu8Xqhzym6P/MVRGuQg9Qu7O5yojzNRxKbZ5+71FYSwGf3t/r/LvOxS/9Xz5z0ncYYY4wxxpspY4wxxpg+eDNljDHGGNMDb6aMMcYYY3rgzZQxxhhjTA/GGs03Z86cSoHfJdLokUceaSonOxKeojdIvZ9FebRGOlHUDEVzSBy51hrlldmprCzKg6JhqE8oKiuLiKCIkC5RSxR5Qu3cuXNnZcuizOh+iq6i9EKStGzZssrWms6BoqMknjs0Tl3Si9AaySLqKJ0D1al13kv1OGfzfiZC6WSydUnznVJYUERW1mc0Xx966KHKlkUktaYvoYiszPc++OCDlY3Wb+bnyVdmc4sgH0SRe11SilA/07rM+pnGlHwApaNq/YyT2lPUSNx+ipKj1FNZKjMau+wzgaDPvtboTon9b2sketZPWXRuC34zZYwxxhjTA2+mjDHGGGN64M2UMcYYY0wPTrmZiogPRMS+iPjOBNvCiPhMRGwe/l2LL4wx5gzAPswYM920CNA/KOn/lPShCbZ3SfpcKeU9EfGu4c+/caoHzZkzpxIiZoKv1rQafQWcJBgmAWX2XBKAZmJvgkR0rSkSsmtJQEpH/2fPbU3bk/UTiS0PHDhQ2bqILXfs2FHZqO8vvvhivH/16tWVjfouE4uTiHvx4sWVjfqZRJ0SC22pnEzkTPOZBKCZAJ1EwSRKpzWSpbgZnftdBKnTyAc1BT6MBOgZNDYkGKaxyYTRtAbvu+++ynbw4EG8n+Y7reHWABCJ53sX/0f+h4TdWZ/QHF60aFFTOVmgUeu6zNYA+bpt27ZVtgceeKCyZT6R2knBMlnaHfJBNB9o3tJ1UvvYU5BCdu1VV11V2bI2UV0J8kFZUE+f9Fen9HSllC9JenjE/FpJtwz/fYukn5t0DYwxZhqxDzPGTDeT/bVxWSnlRObLPZLquHFjjDlzsQ8zxkwZvd/Bl8G75/QwmYi4KSLujIg7KcO4McacTp7Jh030X12+ljbG/HAx2c3U3ohYLknDv/mkREmllPeVUq4vpVxPWh5jjDkNNPmwif4r074ZY8xkT0D/pKS3SHrP8O9PtNwUEZWQMTtFlgSHJDgjAXrm9EjASb9tZiLkjRs3VjY6sbX1VHGJxY4kAs4EcyQWpxNj161bh/e3nvZLZGK9LVu2VLZ77723smWiVoJOO6cTdLuIQukE3/Xr1+P9dAI6tZ/mDo2nxALYvic1t56qLnH/tYp3M1HoaF2zwIkzgM4+bO7cuVVwB62/E9eOQuNAY5CJ9iljwP79+ytb9gaN7OQTaV5m7aS5vWrVqsqWBavQ3KbyM7ExBTCR/ydb1s+0hmicMp9M/UzX0snimSie5g4FGmQ+mfwi+TQaz6yfdu/eXdkoKCjLSnHllVdWNhLaZz6E6kVjR3Osi09speVohA9LukPSFRGxMyLeqoEDelVEbJb0yuHPxhhzxmEfZoyZbk75aqCU8obkv14xxXUxxpgpxz7MGDPdnBGHwBhjjDHGzFS8mTLGGGOM6cFkBeiTopRSCRkzYSKdrpoJcUfJBHN0NAOdrE2iTolF5CQ2pxNfM7E1iaC7iFJJBEiCPxLKSyyCbBWGkyBW4v4jW+tp0pJ05MiRJtvSpUvx/uuuu66yUT/dddddeP+GDRsq26tf/erKRuN06NAhfCbNHep7CrLIrqVxzsTDJIqlMaVnZmM3KgA9Q05AnxLmzp1bCZkzETIJpsl/kIA683MUhEGBFXQyf3YtlU/zMhtHqiv1CQmLJZ5HNN8yATrVq/UU70zsTeuFbNm6JHEzlUV9kn0e0mcClZ/5ZBLFk09u/TySWOxOdbriiivw/jVr1lQ26pPsBPYuYzLKaRGgG2OMMcaYHG+mjDHGGGN64M2UMcYYY0wPvJkyxhhjjOmBN1PGGGOMMT0YazSfVEdfUDoDiY+QJ6U+3Z+lFNm7d29lo4iG7Pj6TZs2VTZKH0LpFJ7//OfjMynyhI7kz6LxWu/ftWsX3n/JJZdUNorQoX7OIpmo/xcuXNhUjsT9TFGXVH4W9UNRKhSNl0WD3HPPPZWNIvx+/Md/vLJl85EiRyi6KqM15VIWtURjStFRXcY+6//ZAKWTydJEUT+0Rr5l/ocisij6aeXKlXg/pR+h+U6RTpmfbo2eylJ8teZrzeYV9RVFuZIt8z/Zehkli3CkCOlWsrEnO0XTZb6GogQpyphsWTQd9SlF6GXRpfTZQ+lsssi71tRZZMvGPot8bsFvpowxxhhjeuDNlDHGGGNMD7yZMsYYY4zpgTdTxhhjjDE9GKsAPSIq4Vcm+CLRG4k1SYTW5Zj/yy+/vLJ95StfwfuJCy+8sLKR0D1LE3DppZdWtta0NxILq5/73OdWtkwsSQJa6lMiE0tSP1922WWVbc+ePc3P/cmf/MmmcrJ0CjTPNm7cWNmoPyXpxhtvbCqLxj5LvUJzp0tABfUTCdi7rDEKdOiSXmR07bWmgJoptKbLoTVEtocffriyZQJmGkcam0ywS2J18ql0f+YTqP2tcyiDROkkTM6e2zcwg9YbrcssKKg1RQ7VnXy/xPWnMcnWOl1LAna6P/OJVH8Kvrrooovwfnou9XP2eU51pflM12XzmQKlWvGbKWOMMcaYHngzZYwxxhjTA2+mjDHGGGN64M2UMcYYY0wPxipAP378eCV6ywSqZCexY6soXZJWrFhR2Uhs+IIXvADvv+KKKyrbtm3bKtu+ffsqGwmTJRagrlu3rrJlYkmyk7AvO4GY+pT6j067zepEJ97ef//9lS0TJl533XVoH4VEndmJytQnNMcy8S4JI0nQS/3ZRajfJSCA7FR+JoCnskgAS326bNkyfOZom2bTieillErMmo1tq/+ieZXNQfIVtAYyUXxrZgPyidkzaW7RmGcnqJNfoTZlonwqi4IwqP5dApWontnYU12pfBI7Z6eFU7aE1tPns/IJalMWPECfKWSjQBuJ5wSJxbMAHPrsp0Av+tw/cOAAPjPLFNKC30wZY4wxxvTAmyljjDHGmB54M2WMMcYY0wNvpowxxhhjeuDNlDHGGGNMD8YazUfRMFmUR2uUC0Ukkcpf4igLigbJIpUoGmb9+vWVjY7pz9KcHDp0qLJRREMW5UGRDhSRtXTpUryfymqN+smgKD0qh9oucURIazqFLPKN5kkWjUNQ/SnqiCKJyCZxm6hOWYoIiqakyJts7lF0GEX9UD2zZ46W32XenOmQ/+qSLoeir44ePdp0ncRrneZLdn/rfKN5na0rinKjOZRFKLaWn91PZdGYtKYZyaB5nEUzk/+i+tNazaKRaewpci2bj9T+1rVOYyzx2NH95HszaEwokjGD2k8R91l0fWsqNcJvpowxxhhjeuDNlDHGGGNMD7yZMsYYY4zpgTdTxhhjjDE9GLsAfVTglYmAW4//J2FgdnQ+XUsiwEceeQTvpzQxJLjrIkqla0lwlwkwSey4YMGCypb1M/UJ9XOreDWzr127trLRMf+S9NBDD1U2GhOqUyYKpXbS/VnqBEqJkF07SiZAp7qSAJLS1kjt4ttMkEwpdlrTTmTXZfN0NhAR1Zhn6XJISEuifZrXmf8hv0ApSbL0G60ialq/WVBPazqvrJ9agyhIlC5xUBHdn9WfaA1sydpEdaWxp/HIPjvIz9P9WcAH1Z/WapfPU4Ke2aXvad1k97fuESjQJgugyT6TWvCbKWOMMcaYHngzZYwxxhjTA2+mjDHGGGN64M2UMcYYY0wPxipAnzNnTiUOpFNcJT51NRPyjpKdYkqnDZO4l0SNEouQCRLR0anoEp8OS+Vnp7KTgJPamYklW0/BJWFhJkCnPqU6delnEv/TMzMBJwkTSdh47rnn4v3Ufipr0aJFlS0TkNM8PXDgQGXrIvSndmaicLqW6kpzNBOljq7R2XYC+uicydYV+TUSwlJ/UwaDE+W3kM03qhP52dZTzSVuf5eTxQm6P/MV5P9a25QJm7OAjT60iriztUq+hp7Zui6l9uCj7POUru1yP40zXZuNB7X14MGDlY3E5tleIvP/LfjNlDHGGGNMD7yZMsYYY4zpgTdTxhhjjDE98GbKGGOMMaYH3kwZY4wxxvRgrNF8Uh2VQJFCEiv96VpS32dRGhR5Qkr/LBqGoswomoTKzyJc6JkUpZVFdFGEEEWudElxQ9ET9MwMupbamaXNoPLp/iVLllS2rJ8p8oPqmaWIIfvy5csrG6V9oKhDiSN3KMoki/Ch+UxlZeuB5hSVRc/MImxWr1590s9dUlHMBEb7J0vdQnN77969lW20v6R8vGhuUP9mvoJ8FUX4taZTycqndCrZPGiN/sr8F0WkkQ+g+7OzKJHNAAAgAElEQVR1ReV38elZX43SJRqvtU+6RM/2jRAkWiP0pPZowqxNNHf3799f2WjsKA1Tdm0rs8vTGWOMMcaMGW+mjDHGGGN64M2UMcYYY0wPTrmZiojVEfGFiLg3Iu6JiHcM7Qsj4jMRsXn494Lpr64xxrRj/2WMGQctquKnJb2zlLIxIs6XdFdEfEbSL0v6XCnlPRHxLknvkvQbz/Sg48ePV2lVuhzd3+WoeYIEayTizVJEkLCT6kSCuUwUSqJSEtpnQtfWFDuZsI5EmCTMpj7Jggeo/SRiJLG2xGNKY0dtyvq5Vai7YAF/ps6fP7+yUd9T2qDHHnsMn0kCUBIJZ2SpmEbJBKCtc4fqn6270YCOTOQ7RqbMfxGHDh1C+/bt2yvb+eefX9lIlJ6tdVqDNA6ZYJjmGwVWdBERt5KlgyFoXWYBMLReWv1HljqE2k8+rYsovzWwpEvqFKJLoFDmK0fJBODk/1t9UteyCOp/8r8UEJEFGmVrr4VTjlApZXcpZePw30clbZK0UtJrJd0yvOwWST836VoYY8w0YP9ljBkHnTRTEbFO0rWSNkhaVkrZPfyvPZIwE29E3BQRd0bEnUeOHOlRVWOMmTx9/Vf2FsoYY5o3UxFxnqS/kvRPSiknHaJSBu/m8P1cKeV9pZTrSynX01lBxhgz3UyF/8q+AjbGmKbNVETM08AR/Xkp5WND896IWD78/+WS9k1PFY0xZvLYfxljpptTqtVioFp8v6RNpZR/O+G/PinpLZLeM/z7E6d6VimlEqhlwro+4tpMhNcqDM+EhfRcqn+XU1xbT+bO+oNOQCdRZib2JhEk1Ymu63KqOp2inYmTDx8+XNmWLl1a2bqcFk5tIkFwJpSlsuiUazpRP5uPVBaNczZ3aJ6SSDkTmpOAn8aJxJok6pTqsTvdAvSp9l+j7aH1J/EcXras/iaRvjrsK7bOhMUkGKY1THMoG0cS/Laeai5JixcvrmyUgSITi9PcbBVhd8mWQP3UZV1R31M/tQaFZHXK+rlV7E5t73KCOc29vqL0LHhq8+bNlY3870UXXVTZMp+6Zs2ajrX7AS2z7iWS3izp2xHxzaHttzRwQn8REW+VtF3Sz0+6FsYYMz3Yfxljpp1TbqZKKV+RlL2CeMXUVscYY6YO+y9jzDjwCejGGGOMMT3wZsoYY4wxpgftx6VOEyQYk1h0RoI5OrGUhMUSC+laxYJSu1idBJzZydYk4CTbpk2b8H4Sa5JYOxOlkjCRbCT0zMSSJIwkQW4mit+yZUtloxOlr7rqqsqWjX2r0D8TW9JJ1STWJmFjJohtPVU6O5Gf1kMXUSutvX376qC25cuXV7ZMFDp6WvrpFqBPJcePH6/8Dc1riQXTNIdIwE6CWYkDFshXZPOq9bTz1hPEJfaV2XwlWk8bz2gN9qH1n4naCXpm1ietmTpo7LKALCqf+rn1pPSM1s9didc23Z/1E409XUs+SZK2bdtW2davX1/ZqJ/oOonHpBW/mTLGGGOM6YE3U8YYY4wxPfBmyhhjjDGmB95MGWOMMcb0wJspY4wxxpgejDWab86cOVVEShb5QRFhpLTvEjlCURat6VQkjhykiAS6P4sSoGgYitzLooauueaaykZtyiJkKC1Ia0RalxQ5lCR2/vz5eP+qVasq22c/+9nKRu288sor8ZlUFqWDyfqZokxoPlHbs9QrVH+KvMsiFEcj5ySeo9kaaY2kovpnEYrf/e53T/q5byqJM4mnn35a+/fvP8nWJVLpwIEDla01wljicaBo3i7psFr9V5c0JbTWKJIwK4ts2Rym+U5Rwl1S5LSui2yc6LkUOUiRnFk/0ziTr8hSfLW2n9qU9RN9dtF6yNL2kP3IkSOVjaL2JE7PRJGwlCIm6+c777wT7S34zZQxxhhjTA+8mTLGGGOM6YE3U8YYY4wxPfBmyhhjjDGmB2MVoJ911llasmTJSbZMWEhC4FbBW5Y6heiSTqZVXEdCz+xI/kOHDlW2Bx54oLKtXLkS728VhmciQCqfhI0kCs3EjnQ/pdOh9BpSe4oNuu473/kOPvOSSy6pbCSKzYTVNE+p/V0E6PRMGrtMAEp1pb7PRNJU/2yejLJ161a0j47JbEon8+STT2rHjh0n2bL2UXAD+RUaw+yZ5KtoDDOxN82t1qAcEkBLLKymtDfZuiJaUz9J7WmusnReBKVZonWR+T+C6kR9l60/KovGKasTjX2rr8k+D1vTWWX+h4IHdu3aVdmydFj0mbh48eLKRgERt912Gz6Tgp9a8ZspY4wxxpgeeDNljDHGGNMDb6aMMcYYY3rgzZQxxhhjTA/GfgL6qOguOxmcBIckQh49kVjKBXMktmwVtWd1ahXxZe0ksfmFF15Y2bLTwrdv317ZSMCZCTBbT9HtItamNh08eLDpmVmdFi1a1FT+nj178JnUT2vXrq1sWT9Rn9C1mVCXoLlHAs4up9dT32UnNZPQlu6nvqMxlmpRaJcMBTOB0bEgobnE85D6lvonO52Z5gGt9UzETPOV5hDNl2yt03yldZ0Jo+m5XYJdKDsAld8lEILa1KVPWrNqUFaI7LOL5k6X0/NbAx2o/Gw+0jyjz86sTbt372565tKlS/F+8rV02vmXv/zlyka+X+JT1VvxmyljjDHGmB54M2WMMcYY0wNvpowxxhhjeuDNlDHGGGNMD7yZMsYYY4zpwVij+aQ8MmAUimhrPao+i+iiSIcu6Tvo2taUIJT6ROIoiwsuuKCyUdSKxHV97LHHKlsWTUgRJVQW3Z/1M0U4rV69urLR0f+SdPTo0cp25MiRykZRO1nU44EDByobRTJRf0gcTdOatiiLsGlNT0RpJ6T2CMOsnnQtRcxS1M3ChQvxmevXrz/p5ywVxEzke9/7njZv3nySbd++fXhta/QWjUEWvUnjTf40i6AkX0Fzq0ualNY0S9kzW1MyZZFzdC3Nd6onpdKSuJ/oma2plyT+7KK2Z2ud7K3pXDI7+SVqe+a/KEqPIoQpRYzEPpk+E7Kxf97znlfZ7r777spGPi377Mki+VvwmyljjDHGmB54M2WMMcYY0wNvpowxxhhjeuDNlDHGGGNMD8YqQC+lVEK4LKVIq1ichLCZ2JuEzUQmQCfBIQlIqe6ZsI2Eha0pFrpcm6VJIWE32VoFlJK0fPnyykbtXLVqFd5PfXXXXXdVNpo7mTCaBPTbtm2rbJlYu1VITWNPokyJ5xkFH2TQemi1SSzKJWEmpfLZsWMHPnN0PnYRM5/pPPbYY9qwYcNJtiywg/qM1iCNQeYrnvOc51Q28knZHCLBMJVP5WRzmMY3EywTJKDvIsym+tOYUN9nwVCtPj0ToNP4Zf57lMz/kJ+nemZzh+zUfvLzWToYmhMUrJIJ/clX09x74QtfiPdv3bq1slGf0DMpDZ2Up65pwW+mjDHGGGN64M2UMcYYY0wPvJkyxhhjjOmBN1PGGGOMMT0YqwB9zpw5Ou+8806yZSJAOsWbxIYkDMxEfFQW2TIBeqsoPhOltpKd+Epkpx2PkgnQqa9aT8HNyibBHwn76ARyidu/ZMmSykbCxqzv6AR2OpmXTuWVVM1biUWx1PZMgEl1pWd2CYig+ZzNRxLF0gnyVD6Nh1TXv1V4OxM4++yzq3lEp/1LLMQl1q1bV9lorkmcFYKExdl8obGgOUTlZ2u9NStFJkqn+Uo+KQtUorJoDVI9s7lJdSIR9sGDB/F+Wm/Up9SmLGCjVaiftYk+p2hMHn/88cpGn8US+08K/sqCgmhOvehFL6psWQAN9T/1KWUpWLt2LT4zyzTSwuzxdMYYY4wxpwFvpowxxhhjeuDNlDHGGGNMD7yZMsYYY4zpwVgF6BFRic4OHz6M15JgkIRoJMJbsWIFPnPnzp1Nz8wgwTCJDbuI2slOIrhMWEhiTaon9Wd2f+sJ6tmp4HQCM5VPJ61L7ac6k/iUBJQZ1M6sn+6///7KdvXVV1c2EmZn4ttMgN9aJ+qn1hOhs2tJOE3jnAk1R/t0Np2AThkcssCOlStXVjaamzSvsnVFzyRxcBaEQWuI5hCtqyyopy+0Nvbs2dN0nZRnURiF/Gx2gjl9JpEtCz6gdd2amSBrJ9lpnLOArtZxJqF9FkxBwm5qe9am66+/vrKRWP3uu+/G+2nuU59ecskllY2CFLLyW/GbKWOMMcaYHngzZYwxxhjTA2+mjDHGGGN6cMrNVEScExFfj4i7I+KeiPjtoX19RGyIiPsj4qMRMT1fqhtjzCSx/zLGjIOWN1Pfl/TyUsrzJb1A0qsj4gZJ75X0h6WUSyUdkvTW6aumMcZMCvsvY8y0c8povjKQxz86/HHe8E+R9HJJbxzab5H0bkk3n+JZVVRBFu1D0SMUqUARWVmk0bJlyyobRSRkdaLoCYqIouirRx99tLJlZe3fv7+yZVGPFH3RGo0nSQsWLKhsFJFB0Q9ZhOHWrVsrG7WpS9odaif16bZt2/B+ihCiebJ+/Xq8nyJCqHwa+yySk6A5nkXztXL06FG0U+oFmic0R7M1MroeTnc031T6r+c85zn6sR/7sZNsW7ZswWv37t1b2SiajtJvbN68GZ+5Zs2aykZRWllEK40t2WitZRGCNL5Z6heiNR1NltKEfA2lbqEoryyamMqifqL0PhJHY5KvbE1PJrFfaI3Qy+zkf+nzkGxSeyoy8p2SdMUVV1S2jRs3VjaKwpf48/zKK6+sbDR2lLYru7aVJs1URMyNiG9K2ifpM5IekHS4lHJiJeyUVMftGmPMacb+yxgz3TRtpkopx0opL5C0StILJdXbv4SIuCki7oyIO7PEkMYYM11Mlf/K3o4YY0ynaL5SymFJX5D0YknzI+LEu99Vkh5K7nlfKeX6Usr1ixYt6lVZY4yZLH39V3bQnzHGtETzLYmI+cN/nyvpVZI2aeCUXje87C2SPjFdlTTGmMlg/2WMGQct6WSWS7olIuZqsPn6i1LKpyLiXkkfiYjflfQNSe8/1YNKKZWQjgTcmZ1EdCTYy0R4y5cvr2wkzu0i4qPfVinNQCaK3759e2X7/Oc/X9kyAfuLX/zipmszAXsrJKrMhNGtotSLLroI7ycB6Y/8yI9UNhKbf/rTn8ZnUtqJxYsXV7b58+fj/ST+pXZS32fpQWiO0zMzoT/NfRKVZqlA1q1bV9lIvExtytJWjM6JLumapokp81/Hjx+v+idLZ0KBHTRfM/9HUBDFrl27KluWTovWK81NspF4XsqF6a3X0dwkX5H55NbUL1R+JpQnX019kt1PY9qaTiYLymkNbMlSt9C1JCynz6Ns7OmZq1evrmzkuyXp61//emXbsWNHZVu6dCnef/nll1c2GjsaJ1pLkvTAAw+gvYWWaL5vSboW7Fs00B8YY8wZif2XMWYc+AR0Y4wxxpgeeDNljDHGGNMDb6aMMcYYY3rQph6cQkaFq5k4lsR5JMQlYWEm2G092Ts7sbpV8Eenq2bPJHHfK1/5ysr24Q9/GO8nweJNN91U2eikZUmis7/oPB0SVWYnLZPgkPrp0ksvxfupT+64447K9rGPfayyZaLQ6667rrLRmCxZsgTvpzGla+nk60yATs/MhN0EzXMSi2ei+tZQfwrSyNbYaFuz62YiTz31VDW+5FMkPkWcxoHuz06cJl9H12bBJhRw0SqAp9OmJQ4ioZPBMz/f6tO7BHGQr6FyqD+ya7usS7qWfM2hQ4cqWyaMJgE61TPrZ5qP9913X/P9BI3zDTfcUNko+4Uk7d69u7LRZ0cmQKc5QfW/9957K9vdd9+Nz7zmmmvQ3sLs8XTGGGOMMacBb6aMMcYYY3rgzZQxxhhjTA+8mTLGGGOM6YE3U8YYY4wxPRhrNF9EVAr8LN0ERYqRUv/JJ5+sbNmR+lTW+vXrKxup/yWOnqDyKUorS71Cz7zkkksq2y/8wi/g/R/5yEcqG6UJecUrXoH3X3311ZVt3rx5lY3SnGSpDyihNaXYyaI8br755sq2adOmyrZ27drKdu211WHXkjjKjyJcqO+k9nQONMeyVBqt13aJeKVIzCyaj+pP85HGM4tOzeb5bGDevHlV9Fq2BsgHUOQd+aosITylyqC5ka0r8qkUTUhRUlk6F4rIojplUbIUUdcadS1xX1H9u0Tj0RymviP/IXFE7YEDByobRVjT54nUP20PlU8+nfqT0ntJ0o/+6I9WNoqupLIl6frrr69sXaIJyVfddtttle2ee+6pbFnUHqWBaq7PpO80xhhjjDHeTBljjDHG9MGbKWOMMcaYHngzZYwxxhjTg7GnkxkVvWUivtb0JSQAzVIkkOCORJ2ZWJLEliT4ozplAkoSRtMz6Zh9iVPPfPzjH69st99+O95/+eWXV7bLLrusslE6ic2bN+MzSRj97W9/u7JRf2blv/Od76xsCxcurGxZ8AGlaSBRbSZAbA10IGF2Nh/JTvMkS6Wxc+fOykZtysS31Ca6lvouE5qP3p+Nx0zkiSeeqMSslFJD4jGjtBiU+iXzia1zI6sTpQUin9hFAE7z/ciRI5UtS5NCQn0SPFNQTFavNWvWVLbWeS3x5wytq2wNkP+jz5nly5dXtiz9Eq2jVqG8xP6bfB199pA/lqQVK1ZUto0bN1Y2arvE9af5mKVC+/KXv1zZtm/fXtkobRkFL0l5W1vwmyljjDHGmB54M2WMMcYY0wNvpowxxhhjeuDNlDHGGGNMD8YqQC+lpCfp0rWjkNiRBIzZKbAkDKf6kFBU4pOFSdjYRRRPAnRqe3b/VVddVdkOHTrUZJP4VGay0QnmmViShIV04uzLXvYyvJ/GNKv/KJn4loSq1KckgJRYREknHVOfZGNHp/1SPbO2k9hy9IRuqV0sLvEao1PVs35qPal5JhIR1fhmImYSDNO8vuCCCypb1oe0BltP9pdYnEzPJLI51BpA0+UU/lbfL3H/U2ALCdi7tCnL1EFQYAyNM41TdqI+tX/v3r2VjXySxGJ3aicJ1UnQL3FWCgoeoFPuJW7/tm3bKtvWrVvxfirrxhtvrGwklF+5ciU+k3xqK34zZYwxxhjTA2+mjDHGGGN64M2UMcYYY0wPvJkyxhhjjOnB2NWi2WnMo5DgjgS7rYJ2iYWRdAp0duL0okWLKhudQk3CPhLxSiyiIxEhtV1iYeF1111X2bLTyqlPqP1UfxJVSizCJmF11ickFqX7SRSeCaNbha5ZP1Of0OnVdH8mKM7KGmXLli1op34mWyZIbhXaUp9movjRsmh9zVQiopqb2diSuJnExTQ2JAKW2udra7CGxOti165dlS0Ta9MapDplwSq0rqhONFclFnu3BoFkonK6nwIKsrlNY0o+ndZVFqxCp4BT8EAWPEXjRD71iiuuqGwkNJfaT3rPBOQkoKf9AQXVSDzPaEwooCrbNxw8eBDtLfjNlDHGGGNMD7yZMsYYY4zpgTdTxhhjjDE98GbKGGOMMaYH3kwZY4wxxvRg7OlkRiMAMlU9qfrpWrquS5oTip6gyA1JWrBgQWWjSAGKKMiizKhNFLmStYnav3r16spGkYgSH99PkSNUp6xNFOVHaV6yKI3WFBUUTZLNp6NHj1Y2Su9Btgwq/9nPfnZly6L2qE2UsiiLxqO60nzIIrEIiq6i6DCKZJTqfu5S9kxgdB1mqTJoDZGvIV+RRblSlB3NjSz1Co0FrWFaQ9l403zJoqEJ8rVU/yxyjiJ/qXxqZxbNR2XR2FHZ2XNb+ySLJjty5Ehlo6jPzH9R1Pjznve8pvKzsSc2btxY2bIUOfSZRNeST5TY11H7X/SiF1W2rE3Z2mnBb6aMMcYYY3rgzZQxxhhjTA+8mTLGGGOM6YE3U8YYY4wxPRirAP3ss8+uxNGZuJYEdySOI7Ff9kw60p9EwHRMv8QiQhLR7dixo7Jl6RBI7NkqoMzsJFbPhImXXHJJZbv00ksrG9U/E+p3SalCUP1J1E5tp3mTlU8BBSQgz55L5ZMoNROq0jh3mc+t4uGs76mfqa60Hnbv3o3PfPzxx0/6OZsjM5VR0WuXNUDi2i6pU1r7cv78+WineUTzjeZFJiImsTylFCGbxP1EczAL4mhNT5YF8BCtovpMVE5Cf+p7ElZnAnRKG0b9lPXzNddcU9kefPDBynbfffc11VPi+Ugpaij4SOI5Rc/MPvtoTlx88cWVrTXwQsr9fwt+M2WMMcYY0wNvpowxxhhjeuDNlDHGGGNMD7yZMsYYY4zpwVgF6HPmzKlEe5kwmkSUK1eurGwkKs9EfHQKLAl2u5w2TiI4EnVmJ6u2is3pBN7sWiorEyGTOI/a33p6fAbVKbufrh0VNksswMyeSSJImnuZyJfKpz6lOZKJd9euXVvZSOyZndZL40QC0Ey8S2JLOimehKp0wrdUj0l2yvRMZM6cOdWcyfwXrWvqMxImZ4JdGlvyf9n9VBadbk9kQRQ0X2heZcJo6j+qfyYMpjVIPoBsmU+mZ5ItEzFTn9Bn0p49eypblhWitZ+uuuoqvH/Dhg2V7dvf/nZla/08k1gUT0E95DslDrSgsjKfvmLFispGfbJ3797K1sUntuI3U8YYY4wxPfBmyhhjjDGmB95MGWOMMcb0oHkzFRFzI+IbEfGp4c/rI2JDRNwfER+NCP4S0hhjTjP2X8aY6aTLm6l3SNo04ef3SvrDUsqlkg5JeutUVswYY6YQ+y9jzLTRFM0XEask/ZSk35P0T2MQxvBySW8cXnKLpHdLuvmZnnPs2LEq+iRLB0BRdhQVQOkMsugDirChKAlKXSJxWg2KFKBnUoSH1B7llkWOUJRJlyP5KUKIIveonlk/U5RZl2iY1nQSVH6WioOupX568skn8X6KhKI6Ud9RH2f30xzrEnVEfZdFqFBfPfTQQ5WN2p5FvI7O3WyMx8lU+a85c+akYzkKzS2KaKPnZelgKBqZ/Ar5RIkjnWi+0jOzcST/vWvXrsqWzRfyyTQvs6hJmu+0LqicLtHIRJe0Zdu2bats9DmTzS+6liL3vva1r+H93/zmNysb9TOVn9WJ5gT5z2zsae7ROGXrgeYzRT7T/X3Hnmh9M/VHkn5d0omVs0jS4VLKid7YKak+t8AYY04/9l/GmGnllJupiPhpSftKKXdNpoCIuCki7oyIO7Pzn4wxZjqYSv+VnRdmjDEtX/O9RNLPRsRrJJ0j6QJJfyxpfkScNfztbpWk+vsBSaWU90l6nyRde+21s+cEP2PMTGDK/NfSpUvtv4wxyCnfTJVSfrOUsqqUsk7S6yV9vpTyJklfkPS64WVvkfSJaaulMcZMAvsvY8w46JNO5jckfSQiflfSNyS9/1Q3PProo7r99ttPsmXiWhJRHjlyBJ85SiYMJLEkieNI/J5BKW5IsJcJOA8cOFDZSGifCeZaUx9kbaLntj4zExaSnQSkJDbM7iexNtU9E/qT2JGe2eV+qmeroFbiOU6C2mzutAr9s0ABEotSO1/ykpdUtixFxHe/+91T1vEMobP/Irq0j66l4AASmmf2VatWVbYsqIf8EvkF+joz89O0hsj/kgBb4vlO/dRFMEzrjfokW5cEtYmE9pJ07733VjYSQZNt4cKF+MylS5dWtq9+9auVbePGjXh/q7Cc+ikbe5o75H+yABjyISQqz8aJ6kV1ovuzr+z7BMx02kyVUr4o6YvDf2+R9MJJl2yMMWPE/ssYM12csb82GmOMMcbMBLyZMsYYY4zpgTdTxhhjjDE96CNA78xTTz1VifYyIVirAJ2EgSSilVhwS+Lc7BRsElGTAJREeNmp6sS+ffsqGwntJRb3kagzEyGT4I7EnvTMDBKltp6WK7Wf6k717HLSMpEJ9an9NB+6tJPm+MUXX1zZNm/ejPfTc0mUmYnFqZ/ppOQ777yzsr32ta/FZ/7Mz/zMST9ngtiZyNy5cyuBcHbaeKvYm/xXF19B843Wn8Rzi6D5ks0hgtq+e/duvPYb3/hGZaPzCLO6X3TRRZWNhPokQs58ausJ5lu3bsX7FyxYUNmoT6iemZ/dsmVLZduwYUNTORL7P/KVZMue2Xr6PNkk/uwiWzb3Wn0y+cls7LMAgBb8ZsoYY4wxpgfeTBljjDHG9MCbKWOMMcaYHngzZYwxxhjTA2+mjDHGGGN6MNZovqeffrqK1Mgi71pTurSm1JDaI/eyyDei9fj5LMrs/PPPbyp///79eD9FJVCUSBbNRtFfWTRQy70Z1E/ZGLeOCUXzZe1sjebJ2k71pyiT1npKHLmybNmyynbdddfh/XfddVdlo7nfpU2PPPJIZTt8+HBl+9a3voXPfM1rXnPSz2dwOpkpIVv/ZG+N/sz6jNYLrXWKfJV4Hq5fv76y0Ry+77778Jm0VilCMYsIo3Ra99xzT2XL0tFQ9BVFQ9K6yKJsab7TWr/66qvxfkoxRpHo5JOytXrHHXegfZQsGrA1co/o8nlIc5xsEn/2U3R/1ib6/CEbld8lOr2V2e3pjDHGGGOmGW+mjDHGGGN64M2UMcYYY0wPvJkyxhhjjOnBaRegZ4K7VnFcqwA8u5YEa1mKCKoTiTpJsJe1h+6/8MIL8VqChOkk4iOxo8QiQBJLdhESkwCV6Cv0J7F5FnxAY0oC1Cy9EQlQScS4dOnSykaicomFsjQemYCdxpRsmfiX1h61M1sPxGibZpMAvZRSicAz/0V9RmNDczjzaTQ2NDeyoJ5W0S0JqI8fP47XPvDAA5WNBPBZPy1atKiykSg9W5ckTG8V9Wd1Iv971VVXVbasP++///7Ktm7duspG9cyE5q2i/sxXkF+kMSWxdvZMspNPpUAbiceU2pQFFbV+HtO6y0Tx2edHC7PH0xljjDHGnAa8mTLGGGOM6YE3U8YYY4wxPfBmyhhjjDGmB2MVoB87dqwSoJMIV2LRGInFSWyZnaxNgucuJxA///nPr2wkQiQBZiYqpbLo2kyUTn0y2sdTAdUzE7pSnej+TNRKIkISi1LfZ3VqPdU4q9P8+fMr28UXX2EAwaIAAAdISURBVFzZVqxYUdmy+dR6In9G6wnwWfABiS3pmXRS/I4dO/CZDz300Ek/Z6dxz0TmzZtXje++ffvw2tYsAq0+KYPmS1Y2+UUqq4uIePXq1ZVty5YtlY3mkMRzk051z9Y13d8qNs98Kq0hqj9lr5Cka6+9trI9/PDDle0LX/hCZcvaSZ+TNCaZWJug9d9F1E7zpDXIS+L+I/+bfZ5TtgbqJxo7CnyQ+vkrv5kyxhhjjOmBN1PGGGOMMT3wZsoYY4wxpgfeTBljjDHG9GCsAvTjx49Xot8u4lgSh5E4LRO8ESROI2GxJN1www2VbfPmzZWtVVSe2Unwl4mYFy5cWNlI1Eqnmmflt4oIsxPMSUTZ5WRZKovupz45evRoczl0MjmdYC6xWJJE6VSnTEBJAlgSlS5ZsgTvX7BgQVNZ2XqgPqX5RKJWOqVakg4dOnTSz5mgdiZCJ6Bnp8tTcASdYk19mwnIKbCEyuly6jz5hS4i5jVr1jRdlwVWkOCY5kwmYCdaxf/Zyf7U/iuuuKKyZevy7rvvrmx0Knom6ieo/6jvs88Z6tMuc4+gzylaD5nvp/XQ6uezssin79y5E+8n+vgrv5kyxhhjjOmBN1PGGGOMMT3wZsoYY4wxpgfeTBljjDHG9MCbKWOMMcaYHow1mq+UUkUWUUoPiaMSKHqMokGylCAU1UQRgtdcc01znej+LtF8RJfUKxTNQ1FeGRQlQ23qkvaidUyyKI3WCCWqE0XYSRylRhF6Xer0xBNPVLbWCBWJUxpQm7KUS5TKY9euXZUtSwdBkTvUf9Smyy67DJ955ZVXnvTzueeei9fNRI4fP16NeZdUQbQGqH+yyDdal13SyZD/o7lB8zqLXKO5sW7duso2GuV5gr1791Y2WqtZlBX5IIpSo8iv5cuX4zNpDVDaoLvuugvvpwhJWsM0H7JIypUrV1Y2GqfM19A8o7lLn8eZn6d5Rv2ctYnGlNZNFjVOc5eeSb47iw6la1vxmyljjDHGmB54M2WMMcYY0wNvpowxxhhjeuDNlDHGGGNMDyITd01LYRH7JW2XtFgS56OYubhNMwO3abysLaWwenmGMcF/SWd2n08Wt+nMZ7a1Rzrz29Tkw8a6mfq7QiPuLKVcP/aCpxG3aWbgNpmpYDb2udt05jPb2iPNnjb5az5jjDHGmB54M2WMMcYY04PTtZl632kqdzpxm2YGbpOZCmZjn7tNZz6zrT3SLGnTadFMGWOMMcbMFvw1nzHGGGNMD8a+mYqIV0fE30bE/RHxrnGXPxVExAciYl9EfGeCbWFEfCYiNg//bk+QdwYQEasj4gsRcW9E3BMR7xjaZ2S7IuKciPh6RNw9bM9vD+3rI2LDcP59NCI4kdkZTETMjYhvRMSnhj/P+DbNFOy/zkxmm/+SZq8Pm63+a6ybqYiYK+k/SLpR0tWS3hARV4+zDlPEByW9esT2LkmfK6VcJulzw59nEk9Lemcp5WpJN0j61eHYzNR2fV/Sy0spz5f0AkmvjogbJL1X0h+WUi6VdEjSW09jHSfLOyRtmvDzbGjTGY/91xnNbPNf0uz1YbPSf437zdQLJd1fStlSSnlS0kckvXbMdehNKeVLkh4eMb9W0i3Df98i6efGWqmelFJ2l1I2Dv99VIPJvlIztF1lwInU4POGf4qkl0v6y6F9xrTnBBGxStJPSfpPw59DM7xNMwj7rzOU2ea/pNnpw2az/xr3ZmqlpB0Tft45tM0GlpVSdg//vUfSstNZmT5ExDpJ10raoBncruHr5G9K2ifpM5IekHS4lPL08JKZOP/+SNKvSzo+/HmRZn6bZgr2XzOA2eK/pFnpw2at/7IAfRoogxDJGRkmGRHnSforSf+klPLIxP+bae0qpRwrpbxA0ioN3ipceZqr1IuI+GlJ+0opd53uupjZy0xb5xOZTf5Lml0+bLb7r7PGXN5DklZP+HnV0DYb2BsRy0spuyNiuQa/ScwoImKeBo7oz0spHxuaZ3y7SimHI+ILkl4saX5EnDX8TWimzb+XSPrZiHiNpHMkXSDpjzWz2zSTsP86g5mt/kuaNT5sVvuvcb+Z+htJlw3V+2dLer2kT465DtPFJyW9Zfjvt0j6xGmsS2eG312/X9KmUsq/nfBfM7JdEbEkIuYP/32upFdpoKP4gqTXDS+bMe2RpFLKb5ZSVpVS1mmwdj5fSnmTZnCbZhj2X2cos81/SbPPh816/1VKGesfSa+RdJ8G3/3+s3GXP0Vt+LCk3ZKe0uA73rdq8N3v5yRtlvRZSQtPdz07tunHNXgF/i1J3xz+ec1MbZek50n6xrA935H0L4b2iyV9XdL9kv6rpGed7rpOsn0vlfSp2dSmmfDH/uvM/DPb/NewTbPWh81G/+UT0I0xxhhjemABujHGGGNMD7yZMsYYY4zpgTdTxhhjjDE98GbKGGOMMaYH3kwZY4wxxvTAmyljjDHGmB54M2WMMcYY0wNvpowxxhhjevD/AwKxH+2yhwq9AAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 720x360 with 2 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.figure(figsize=[10,5])\n",
    "\n",
    "# Display the first image in training data\n",
    "plt.subplot(121)\n",
    "plt.imshow(np.squeeze(X_train[25,:,:], axis = 2), cmap='gray')\n",
    "plt.title(\"Ground Truth : {}\".format(get_label(int(y_train[0]))))\n",
    "\n",
    "# Display the first image in testing data\n",
    "plt.subplot(122)\n",
    "plt.imshow(np.squeeze(X_test[26,:,:], axis = 2), cmap='gray')\n",
    "plt.title(\"Ground Truth : {}\".format(get_label(int(y_test[1500]))))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# VI. Emotional Deep Alignment Networks"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This is an implementation of the paper : https://arxiv.org/pdf/1810.10529.pdf\n",
    "\n",
    "The main advantage of DAN over the competing\n",
    "face alignment methods comes from an iterative process\n",
    "of adjusting the locations of facial landmarks. The iterations are incorporated into the neural network architecture, as the information about the landmark locations detected in the previous stage (layer) are transferred to the\n",
    "next stages through the use of facial landmark heatmaps.\n",
    "As a result and contrary to the competing methods, DAN\n",
    "can therefore handle entire face images instead of patches\n",
    "which leads to a significant reduction in head pose variance\n",
    "and improves its performance on a landmark recognition\n",
    "task. \n",
    "\n",
    "Even though during training model has only information about emotion label, spatial information about\n",
    "regions containing information relevant to expressed emotions are correctly captured. Face regions around\n",
    "eyes and mouth influence model’s decision the most for all emotion classes. Eyebrow location and strength of\n",
    "activations near nose (e.g Disgust) also show some discriminative information."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-03-06T07:47:00.624036Z",
     "start_time": "2019-03-06T07:47:00.601287Z"
    }
   },
   "outputs": [],
   "source": [
    "import math\n",
    "\n",
    "from tensorflow.python.eager import context\n",
    "from tensorflow.python.framework import constant_op\n",
    "from tensorflow.python.framework import dtypes\n",
    "from tensorflow.python.framework import ops\n",
    "from tensorflow.python.ops import control_flow_ops\n",
    "from tensorflow.python.ops import math_ops\n",
    "from tensorflow.python.ops import random_ops\n",
    "from tensorflow.python.util.tf_export import tf_export\n",
    "\n",
    "from skimage.transform import resize\n",
    "from skimage import color\n",
    "import tensorflow as tf"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-03-06T07:47:48.366101Z",
     "start_time": "2019-03-06T07:47:48.361758Z"
    }
   },
   "outputs": [],
   "source": [
    "def e_distance(a, b):\n",
    "    return np.linalg.norm(a - b)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-03-06T07:47:48.748527Z",
     "start_time": "2019-03-06T07:47:48.739148Z"
    }
   },
   "outputs": [],
   "source": [
    "def find_closest(matrix_list, target_matrix):\n",
    "    min_distance = 999999\n",
    "    closest_matrix = None\n",
    "    closest_id = None\n",
    "    for i, m in enumerate(matrix_list):\n",
    "        if len(m) > 0:\n",
    "            dist = e_distance(m, target_matrix)\n",
    "            if dist < min_distance:\n",
    "                closest_matrix = m\n",
    "                closest_id = i\n",
    "                min_distance = dist\n",
    "    return closest_matrix, closest_id"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-03-06T07:47:50.271110Z",
     "start_time": "2019-03-06T07:47:50.266728Z"
    }
   },
   "outputs": [],
   "source": [
    "emotionDict7 = {0: 'Neutral', 1: 'Happiness', 2: 'Sadness',\n",
    "                3: 'Surprise', 4: 'Fear', 5: 'Disgust', 6: 'Anger'}\n",
    "\n",
    "IMGSIZE = 2304"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-03-06T07:48:30.691469Z",
     "start_time": "2019-03-06T07:48:30.625970Z"
    }
   },
   "outputs": [],
   "source": [
    "def get_gradcam(batch_img, batch_label, modelPath, dan, tf_sess,\n",
    "                img_mask=1, img_size=IMGSIZE, conv_layer='S2_Conv4a', logging=False):\n",
    "    # TODO: fix for batch size > 1\n",
    "\n",
    "    images = tf.placeholder(tf.float32, [None, img_size, img_size, 1])\n",
    "    labels = tf.placeholder(tf.float32, [None, ])\n",
    "\n",
    "    # gradient for partial linearization. We only care about target\n",
    "    # visualization class.\n",
    "    y_c = tf.reduce_sum(tf.multiply(dan['S2_Emotion'], labels), axis=1)\n",
    "\n",
    "    # Get last convolutional layer gradient for generating gradCAM\n",
    "    # visualization\n",
    "    target_conv_layer = dan[conv_layer]\n",
    "    target_conv_layer_grad = tf.gradients(y_c, target_conv_layer)[0]\n",
    "\n",
    "    # Run inference of prediction class\n",
    "    prob, softmax = tf_sess.run([dan['Pred_emotion'], dan['softmax']],\n",
    "                       {dan['InputImage']: batch_img,\n",
    "                        dan['S1_isTrain']: False,\n",
    "                        dan['S2_isTrain']: False})\n",
    "    if logging:\n",
    "        # print(prob)\n",
    "        # print(softmax)\n",
    "        print('Predicted emotion:', emotionDict7[prob[0]])\n",
    "        print('True emotion', emotionDict7[batch_label[0]])\n",
    "        for i in range(len(softmax[0])):\n",
    "            print(emotionDict7[i], softmax[0][i])\n",
    "        \n",
    "\n",
    "    # Do not generate cam if predicted class is not correct\n",
    "    if prob[0] != batch_label[0]:\n",
    "        return []\n",
    "\n",
    "    target_conv_layer_value, target_conv_layer_grad_value = tf_sess.run(\n",
    "        [target_conv_layer, target_conv_layer_grad],\n",
    "        feed_dict={images: batch_img,\n",
    "                   labels: batch_label,\n",
    "                   dan['InputImage']: batch_img,\n",
    "                   dan['S1_isTrain']: False,\n",
    "                   dan['S2_isTrain']: False})\n",
    "\n",
    "    batch_size = len(batch_img)\n",
    "\n",
    "    for i in range(batch_size):\n",
    "\n",
    "        output, grads_val = target_conv_layer_value[i], target_conv_layer_grad_value[i]\n",
    "        weights = np.mean(grads_val, axis=(0, 1))\n",
    "\n",
    "        cam = np.zeros(output.shape[0: 2], dtype=np.float32)\n",
    "        for j, w in enumerate(weights):\n",
    "            cam += w * output[:, :, j]\n",
    "\n",
    "        # Passing through ReLU\n",
    "        cam = np.maximum(cam, 0)\n",
    "        cam = cam / np.max(cam)  # scale 0 to 1.0\n",
    "        cam = resize(cam, (img_size, img_size), preserve_range=True)\n",
    "\n",
    "        return cam"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-03-05T14:32:39.802969Z",
     "start_time": "2019-03-05T14:32:39.797214Z"
    }
   },
   "source": [
    "# VII. Save the dataframes"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-03-05T14:41:45.625325Z",
     "start_time": "2019-03-05T14:41:43.527770Z"
    }
   },
   "outputs": [],
   "source": [
    "np.save(local_path + 'X_train_augmented', train_data)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-03-05T14:41:39.801541Z",
     "start_time": "2019-03-05T14:41:39.541607Z"
    }
   },
   "outputs": [],
   "source": [
    "np.save(local_path + 'X_test_augmented', test_data)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-03-05T14:41:40.486614Z",
     "start_time": "2019-03-05T14:41:40.475886Z"
    }
   },
   "outputs": [],
   "source": [
    "np.save(local_path + 'y_train_augmented', train_labels_one_hot)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-03-05T14:41:41.305668Z",
     "start_time": "2019-03-05T14:41:41.300922Z"
    }
   },
   "outputs": [],
   "source": [
    "np.save(local_path + 'y_test_augmented', test_labels_one_hot)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# VIII. Sources"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- Visualization : https://github.com/JostineHo/mememoji/blob/master/data_visualization.ipynb\n",
    "- State of the art Architecture : https://github.com/amineHorseman/facial-expression-recognition-using-cnn\n",
    "- Eyes Tracking : https://www.pyimagesearch.com/2017/04/24/eye-blink-detection-opencv-python-dlib/\n",
    "- Face Alignment : https://www.pyimagesearch.com/2017/05/22/face-alignment-with-opencv-and-python/\n",
    "- C.Pramerdorfer,  and  M.Kampel.Facial  Expression  Recognition  using  Con-volutional  Neural  Networks:  State  of  the  Art.  Computer  Vision  Lab,  TU  Wien. https://arxiv.org/pdf/1612.02903.pdf\n",
    "- A Brief Review of Facial Emotion Recognition Based\n",
    "on Visual Information : https://www.mdpi.com/1424-8220/18/2/401/pdf\n",
    "- Going deeper in facial expression recognition using deep neural networks : https://ieeexplore.ieee.org/document/7477450\n",
    "- Emotional Deep Alignment Network paper : https://arxiv.org/abs/1810.10529\n",
    "- Emotional Deep Alignment Network github : https://github.com/IvonaTau/emotionaldan"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.5"
  },
  "latex_envs": {
   "LaTeX_envs_menu_present": true,
   "autoclose": false,
   "autocomplete": true,
   "bibliofile": "biblio.bib",
   "cite_by": "apalike",
   "current_citInitial": 1,
   "eqLabelWithNumbers": true,
   "eqNumInitial": 1,
   "hotkeys": {
    "equation": "Ctrl-E",
    "itemize": "Ctrl-I"
   },
   "labels_anchors": false,
   "latex_user_defs": false,
   "report_style_numbering": false,
   "user_envs_cfg": false
  },
  "toc": {
   "base_numbering": 1,
   "nav_menu": {},
   "number_sections": true,
   "sideBar": true,
   "skip_h1_title": false,
   "title_cell": "Table of Contents",
   "title_sidebar": "Contents",
   "toc_cell": false,
   "toc_position": {},
   "toc_section_display": true,
   "toc_window_display": false
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
